Over the last few decades, internet use has increased dramatically, particularly with the emergence of social networks. However, this explosion in social media activity has led to the spread of harmful messages, including hate speech. Hate speech is a type of degrading communication aimed specifically at an individual or group, that can take the form of threats and acts of violence. Deep learning models have quickly become a means of detecting hate speech. However, these models require a substantial amount of training data to achieve high levels of performance. Some features can be effective in neural networks to overcome the problem of limited training data for hate speech detection. In this thesis, we examine two main areas of research aimed at improving hate speech detection performance. We test our methods on four separate datasets, allowing us to thoroughly evaluate their effectiveness in detecting hate speech. In the first part of our study, we propose to enhance the hate speech detection by incorporating specific features into a neural network based on sentence embeddings. More specifically, we examine the incorporation of features such as word case, emojis, words present in a hate speech lexicon, part-of-speech, and punctuation. Our approach involves developing a neural network that integrates these features at the word level, in addition to sentence embeddings. We demonstrate that the use of emojis significantly improves the performance of hate speech detection. Next, we focus on the integration of underexplored features in hate speech detection: multiword expressions. We conduct an in-depth study on the robustness of systems for identifying these expressions in tweets. This study allows us to assess different systems for identifying multiword expressions in tweets, in order to automatically annotate datasets intended for hate speech detection in terms of multiword expressions. We demonstrate that deep learning-based systems outperform dictionary-based ones in this task. Furthermore, we propose a two-step system that combines both the deep learning-based and dictionary-based systems. This system outperforms the two existing systems for identifying multiword expressions in tweets. Then, we develop two neural networks that rely on sentence embeddings and integrate these multiword expressions in different ways. We show a significant improvement in performance on the hate speech detection task using multiword expression information. In the second part, we explore different learning approaches to enhance performance in hate speech detection. First, we investigate the impact of multi-task learning. For this purpose, we propose a multi-head attention-based neural network for multi-task learning. Our system is designed to simultaneously learn two tasks: hate speech detection and multiword expression identification. We demonstrate that jointly learning attention mechanisms for multiword expressions and hate speech enhances the detection of the latter. Next, we explore the use of contrastive learning for hate speech detection. Our approach involves applying this learning method in a supervised manner. The main objective is to learn sentence embeddings such that tweets belonging to the same class are brought closer together (according to cosine similarity), while tweets from different classes are pushed apart. To achieve this goal, we propose different methods to create pairs of training tweets. We also suggest various cosine similarity-based decision methods to predict classes. We demonstrate that our approaches achieve performance equivalent to traditional learning for hate speech classification. However, the intriguing aspect of our approach lies in the fact that the sentence embeddings generated through contrastive learning lead to a better separation of classes in an observable vector space, compared to embeddings generated through conventional learning for hate speech classification.