Skip to Main content Skip to Navigation
Theses

La construction automatique de ressources multilingues à partir des réseaux sociaux : application aux données dialectales du Maghreb

Karima Abidi 1
1 SMarT - Statistical Machine Translation and Speech Modelization and Text
LORIA - NLPKD - Department of Natural Language Processing & Knowledge Discovery
Abstract : Automatic language processing is based on the use of language resources such as corpora, dictionaries, lexicons of sentiments, morpho-syntactic analyzers, taggers, etc. For natural languages, these resources are often available. On the other hand, when it comes to dealing with under-resourced languages, there is often a lack of tools and data. In this thesis, we are interested in some of the vernacular forms of Arabic used in Maghreb. These forms are known as dialects, which can be classified as poorly endowed languages. Except for raw texts, which are generally extracted from social networks, there is not plenty resources allowing to process Arabic dialects. The latter, compared to other under-resourced languages, have several specificities that make them more difficult to process. We can mention, in particular the lack of rules for writing these dialects, which leads the users to write the dialect without following strict rules, so the same word can have several spellings. Words in Arabic dialect can be written using the Arabic script and/or the Latin script (arabizi). For the Arab dialects of the Maghreb, they are particularly impacted by foreign languages such as French and English. In addition to the borrowed words from these languages, another phenomenon must be taken into account in automatic dialect processing. This is the problem known as code- switching. This phenomenon is known in linguistics as diglossia. This gives free rein to the user who can write in several languages in the same sentence. He can start in Arabic dialect and in the middle of the sentence, he can switch to French, English or modern standard Arabic. In addition to this, there are several dialects in the same country and a fortiori several different dialects in the Arab world. It is therefore clear that the classic NLP tools developed for modern standard Arabic cannot be used directly to process dialects. The main objective of this thesis is to propose methods to build automatically resources for Arab dialects in general and more particularly for Maghreb dialects. This represents our contribution to the effort made by the community working on Arabic dialects. We have thus produced methods for building comparable corpora, lexical resources containing the different forms of an input and their polarity. In addition, we developed methods for processing modern standard Arabic on Twitter data and also on transcripts from an automatic speech recognition system operating on Arabic videos extracted from Arab television channels such as Al Jazeera, France24, Euronews, etc. We compared the opinions of automatic transcriptions from different multilingual video sources related to the same subject by developing a method based on linguistic theory called Appraisal.
Complete list of metadata

Cited literature [139 references]  Display  Hide  Download

https://hal.univ-lorraine.fr/tel-02510812
Contributor : Thèses Ul <>
Submitted on : Wednesday, March 18, 2020 - 11:01:49 AM
Last modification on : Tuesday, December 8, 2020 - 10:12:36 AM
Long-term archiving on: : Friday, June 19, 2020 - 12:57:46 PM

File

DDOC_T_2019_0274_ABIDI.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : tel-02510812, version 1

Citation

Karima Abidi. La construction automatique de ressources multilingues à partir des réseaux sociaux : application aux données dialectales du Maghreb. Informatique [cs]. Université de Lorraine, 2019. Français. ⟨NNT : 2019LORR0274⟩. ⟨tel-02510812⟩

Share

Metrics

Record views

196

Files downloads

618