Skip to Main content Skip to Navigation

Reconnaissance et traduction automatique de la parole de vidéos arabes et dialectales

Mohamed Amine Menacer 1, 2 
1 SMarT - Statistical Machine Translation and Speech Modelization and Text
LORIA - NLPKD - Department of Natural Language Processing & Knowledge Discovery
2 MULTISPEECH - Speech Modeling for Facilitating Oral-Based Communication
Inria Nancy - Grand Est, LORIA - NLPKD - Department of Natural Language Processing & Knowledge Discovery
Abstract : This research has been developed in the framework of the project AMIS (Access to Multilingual Information and opinionS). AMIS is an European project which aims to help people to understand the main idea of a video in a foreign language by generating an automatic summary of it. In this thesis, we focus on the automatic recognition and translation of the speech of Arabic and dialectal videos. The statistical approaches proposed in the literature for automatic speech recognition are language independent and they are applicable to modern standard Arabic. However, this language presents some characteristics that we need to take into consideration in order to boost the performance of the speech recognition system. Among these characteristics we can mention the absence of short vowels in the text, which makes their training by the acoustic model difficult. We proposed several approaches to acoustic and/or language modeling in order to better recognize the Arabic speech. In the Arab world, modern standard Arabic is not the mother tongue, that is why daily conversations are carried out with dialect, an Arabic inspired from modern standard Arabic, but not only. We worked on the adaptation of the speech recognition system developed for the modern standard Arabic to the Algerian dialect, which is one of the most difficult variants of the Arabic language to recognize by automatic speech recognition systems. This is mainly due to the borrowed words from other languages, the code-switching and the lack of resources. Our approach to overcome all these problems is to take advantage from oral and textual data of other languages that have an impact on the dialect in order to train the required models for dialect speech recognition. The resulting text from Arabic speech recognition system was then used for machine translation. As a starting point, we conducted a comparative study between the phrase based approach and the neural approach used in machine translation. Then, we adapted these two approaches to translate the code-switched text. Our study focused on the mix of Arabic and English in a parallel corpus extracted from official documents of the United Nations. In order to prevent the error propagation in the pipeline system, we worked on the adaptation of the vocabulary of the automatic speech recognition system and on the proposition of a new model that directly transforms a speech signal in language A into a sequence of words in another language B.
Document type :
Complete list of metadata
Contributor : Thèses UL Connect in order to contact the contributor
Submitted on : Friday, February 5, 2021 - 3:02:50 PM
Last modification on : Saturday, June 25, 2022 - 7:43:50 PM
Long-term archiving on: : Friday, May 7, 2021 - 8:27:39 AM


Files produced by the author(s)


  • HAL Id : tel-03132934, version 1


Mohamed Amine Menacer. Reconnaissance et traduction automatique de la parole de vidéos arabes et dialectales. Informatique et langage [cs.CL]. Université de Lorraine, 2020. Français. ⟨NNT : 2020LORR0157⟩. ⟨tel-03132934⟩



Record views


Files downloads