Full Text Article

Introduction

The extensive volume of opinions and reviews posted online by social media users regarding policies, services, and products highlights the importance of understanding the vital information contained in social media content. This understanding is critical for a variety of stakeholder groups, including customers, business owners, and investors. Individuals use social media for many reasons, one of which is to express their opinions about products and political issues. This activity encourages various parties, such as consumers, businesses, and government entities, to participate in analyzing these opinions. Indeed, paying attention to customer feedback and reviews is a key tool in influencing decision-making processes. For organizations and individuals to improve their products and services, it is essential to uncover the range of sentiments conveyed and then use this information to formulate recommendations that are tailored to the unique needs of customers [1].

Emotion analysis (EA) is a subtask of natural language processing (NLP) that aims to analyze big data to discover people's opinions and emotions. Emotion analysis, or the detection of more complex feelings, is a relatively new field that presents new challenges in addition to those faced by sentiment analysis, where emotion analysis is a better classification [2-4]. Existing multi-label text-based emotion analysis Arabic datasets suffer from a high level of class imbalance. Where the number of cases in a certain class is very high, while in other classes the number of cases is low. In real life, the distribution of examples (training tweets) is biased since comments belonging to certain emotion classes rarely appear. This presents a difficulty for learning algorithms because they are biased towards the majority of classes. However, most text-based Arabic emotion analysis work assumes balanced sample sizes for each emotion class, which is not in accordance with reality [4-6]. The application of supervised learning in Arabic sentiment analysis is hampered by the insufficient datasets containing multilabel sentiment annotations, which are limited in size and imbalanced. Consequently, supervised learning techniques designed for balanced classification struggle to deliver desired results when faced with imbalanced data, negatively impacting the overall performance of sentiment analysis. Moreover, there has been a paucity of research addressing the challenges posed by imbalanced class distribution in the field of sentiment analysis [7-10]. There is also a lack of in- -depth study on the impact of imbalanced classes in Arabic sentiment analysis. This work addresses the problem of class imbalance, which is one of the most difficult problems in multi-sentence analysis describing text-based Arabic. Moreover, sentiment analysis of Arabic is still in its infancy; in fact, researchers do not cover many dialects and there are few sentiment resources, which discourages field research from balancing the work done in other languages such as English [11]. The objective of this paper is to develop an improved model for Arabic text-based sentiment analysis with imbalanced datasets. In addition, to design an ensemble framework for heterogeneous learning models for Arabic textbased sentiment analysis. The rest of this paper is organized as follows. Section 2 presents related work while Section 3 presents the proposed methodology. The experimental setup is described in Section 4. Section 5 talks about the outcomes of the experiment. In Section 6, we finally wrap up our findings and explore potential avenues for further research.

Related Work

Emotion analysis can be conducted through various methodologies and has numerous applications. The primary techniques include lexicon-based, machine-learning-based, and deep learning-based approaches for emotion analysis. A comprehensive survey detailing research efforts in emotion analysis, the techniques employed, and the resources available are presented in [12]. Additionally, a systematic review focusing on the applications of natural language processing and the future challenges, particularly in text-based emotion detection, is discussed in [13]. The study in [14], examines several machine learning (ML) algorithms, including naive Bayes, support vector machines, and decision trees (DT), applied to sentiment analysis of airline review datasets. Furthermore, [15] offers a systematic review of machine-learning-based text classification methods. The research in [16] employs decision trees (DT), support vector machines (SVM), artificial neural networks (ANN), K-Nearest Neighbors (KNN), and Naïve Bayes (NB), along with ensemble models such as random forest (RF) and gradient boosting (GB), which utilize bagging and boosting techniques, as well as three sampling strategies based on ensemble hybrid sampling for addressing imbalanced data.

A study referenced in [17] proposed a method for classifying emotions in Arabic tweets utilizing a deep Convolutional Neural Network (CNN). This deep learning architecture functions as an end-to-end network, incorporating steps for word, sentence, and document vectorization. In [6], they used optimization BiLSTM network for multilabel Arabic emotion analysis and employed a CBOW word embedding model for word representation. A full survey about emotion detection in Arabic text in social media is introduced in [18]. The research presented in [19] uses bidirectional encoder representation by transformer models (BERT) for sentiment analysis and emotion recognition of Twitter data. In [20], they present an automatic text annotation methodology to label Arabic text data as multi-labels based on sets of extracted key phrases. They used to reduce the size of the features with the vector representation of the Bi-gram alphabet to build the document vectors. In [5], they present a model based on three state-of-the-art deep learning models. Two models are special types of recurrent neural networks RNN (Bi-LSTM and Bi-GRU), and the third model is a preformed linguistic model (PLM) based on BERT.

The methodology presented in [21] comprises three primary components: an embedding layer for word representation, a Bi-LSTM framework for capturing both forward and backward contextual information, and a sigmoid layer for classification aimed at emotion recognition, focusing on five core emotions: joy, sadness, fear, shame, and guilt. In [22], the authors introduced Enhanced Long Short-Term Memory (ELSTM) to identify emotions within Twitter data. The study in [23] employed LSTM, SVM, and nested LSTM techniques to classify multiple emotion labels successfully. In [24], they classified emotions into seven they are: (fear, anger, love, joy, surprise, thankfulness and sadness) using LSTM AND nested LSTM. In [25], they used method naïve Bayes, support vector machines, artificial neural network (ANN), and recurrent neural network (RNN). The study referenced in [26] implemented a multi-head attention mechanism combined with bidirectional long short- -term memory and convolutional neural networks (MHABCNN). In [27], the researchers applied multiple methods for analyzing emotions in Arabic text, including bidirectional GRU_CNN (BiGRU_CNN), conventional neural networks (CNN), and an XGBoost regressor (XGB). They gathered a dataset of tweets by utilizing the Twitter API and conducting searches using emotion-related keywords. The findings displayed a Pearson coefficient of 69.2%.

In [28], they combine an attention-based LSTM-- BiLSTM deep model with the transformer-based pre-- trained Arabic Bidirectional Encoder Representations from the Transformers model (AraBERT ) for Arabic language understanding to address the issue of Arabic affect detection (multi-label emotion categorization). The label-emotion of tweets is determined by the attention-based LSTM-BiLSTM, whereas AraBERT creates the contextualized embedding. Their suggested strategy performs better than the eight baseline techniques. It obtains a noteworthy accuracy rate of 53.82% on the SemEval2018-Task1.

Methodology

In this study, we employed a comprehensive methodology that encompasses all essential steps for the detection and analysis of emotions. The proposed approach consists of multiple phases, including preprocessing, management of unbalanced classes, feature selection, and emotion analysis, as illustrated in Figure 1. The primary objective of this paper is to develop an ensemble framework for analyzing emotions in imbalanced Arabic text.

Pre-Processing Phase

The reviews and information collected from social media platforms and websites are inherently unstructured, similar to other forms of user-generated content, which complicates the analysis of sentiments. These datasets often contain errors such as misspellings, abbreviations, repeated characters, special symbols, and HTML tags. Therefore, it is essential to preprocess this data before any further analysis can take place. This preprocessing phase can also be regarded as a dimensionality reduction step, as it standardizes different word forms and eliminates irrelevant stop words, including prepositions, conjunctions, and articles, which do not influence sentiment and are commonly present in reviews and opinion pieces. Figure 2. displays the preprocessing methods utilized in this research encompassed tokenization, normalization, stop word removal, and stemming.

Normalization

Datasets on emotions that are gathered from social media sites are invariably unstructured and noisy. In social media, user-generated material is inherently casual, includes emoticons and emojis, and is frequently misspelled. Sometimes English words and special characters appear in Arabic evaluations. All HTML, links, and programming language code are deleted from the text along with certain English words and special characters. The second stage, normalization, transforms various Arabic word forms into a have led to the various forms of Arabic words. For instance, takenly thought of as three distinct terms. They must all be changed into one of these representations as a result with uniformity of shape.

Tokenization

Tokenization stage is a vital step in any text mining process. Texts are divided into sentences, which are subsequently divided into lists of words or n-grams. A review is sent into the tokenization process, which transforms it into a representation in the form of a bag of words or bag of n-grams. The n-gram representation bigrams, trigrams, and unigrams is used in this work to express the meaning of valued emotions in phrases. To indicate the borders of words and sentences (or major tokens), punctuation marks and white spaces are utilized [29].

Stop Word Removal

In every language, stop words are the most prevalent and frequently nonsemantic expressions. Reviews contain stop words like determiners, prepositions, and pronouns, just as other types of writing. While certain stop words mostly negations are helpful in analyzing emotions, others are not. There is often a list of stop words in every language, such terms were eliminated since they are regularly seen in literature from all classes and do not contribute to class discrimination.

Stemming

This step is frequently called stemmers. A computational process known as stemming lowers all words that have the same root (or stem, if prefixes are omitted) to a common form. We used Root-based approach to do this. Typically, this is accomplished by depriving each word of it sufixes that are derivational and inflectional. By using a stemming method, the words are boiled down to their root. Here, our goal is to distill a word's various incarnations to its essential root or stem. This is useful in the field of information retrieval (IR) since it makes managing words with similar basic meanings more convenient. In information retrieval, matching documents to a query becomes more successful when terms with the same root (or stem) are grouped together. For the purposes of IR, a basic stemming of the English language that entails the removal of sufixes is enough. However, removing sufixes by alone would not be adequate for Arabic. Antefixes, prefixes, sufixes, and postfixes are the four types of affixes that can be added to words in Arabic [30, 31].

Handling Imbalanced Class

The process of balancing a dataset using SMOTE, which stands for Synthetic Minority Oversampling Technique, involves the generation of synthetic data points for the minority class to achieve equilibrium within the dataset. This is accomplished by augmenting the minority class data through the creation of new synthetic instances derived from the existing data. The methodology employs a KNN algorithm to facilitate the generation of these synthetic data points [32,33]. In this study, the over-sampling strategy is implemented to ensure a balanced dataset. SMOTE effectively addresses the challenges posed by unbalanced datasets. This technique involves over-sampling the minority classes by generating synthetic samples that are based on the similarities in feature space among the existing minority instances. A vector is formed between the current data point and one of its KNNs, which is then multiplied by a randomly generated integer between 0 and 1 to produce a new synthetic data point. The steps involved in the SMOTE algorithm are outlined succinctly.

1- Identify the minority class vector.

2- Decide the number of nearest numbers (k), to consider.

3- Compute a line between the minority data points and any of its neighbors and place a synthetic point.

4- Repeat step 3 for all minority data points and their k neighbors, till the data is balanced.

Feature Selection Phase The curse of dimensionality, wherein the number of created features is disproportionately large, is one of the most important issues with text mining jobs. Feature selection, also known as dimensionality reduction, is one of the most crucial stages in emotion analysis, during this process only discriminating characteristics are chosen. Many characteristics still lack discriminative power after the preprocessing and data representation stages. Only a small portion of the very large characteristics that were gathered include information that is useful for emotion analysis. An appropriate feature selection strategy that minimizes feature size is therefore required. The act of choosing the lowest subset of features to use in an analysis to minimize dimensionality while maintaining acceptable analysis performance is known as feature selection.

Filtering-based methods assign weights to features based on their effectiveness in differentiating between classes, as indicated by the data representation matrix. These weights help assess the degree of association between the features and the target class. A feature with a high weight indicates its potential utility in classication tasks. e features are maintained in a ranked list, organized in descending order of importance. Only the top n rated characteristics are chosen. The chi-squared statistic (χ2) is one of the most popular FS, it was utilized in this study and it is effective for emotion analysis. The χ 2 statistic is one commonly used feature selection. Chi-square estimates whether the class label is independent of a feature. The chi-square score with class c and feature/word w is defined as:

where A is the number of times that w and c co-occur, B is the number of times that w occurs without c, C is the number of times that c occurs without w, D is the number of times that neither c nor w occurs, and N is the total number of reviews[34,35].

This section outlines the proposed ensemble learning model, which incorporates the classification algorithms SVM, NV, and KNN. The subsequent subsection provides a detailed description of these algorithms. The comprehensive ensemble framework is illustrated in Figure 3.

Naive Bayes (NB)

The algorithm determines the posterior probability given a representation matrix and gives the review of the class with the largest posterior probability. The main benefit of NB algorithms is that, in many cases, they perform better and are simple to implement. The NB binary classifiers solve the emotion analysis problem given a review which is represented as a set of feature terms and is a class in the class set [36,37]. Naive Bayes (NB) can be defined as the conditional probability of a given constructed as in Equation(2):

Thus, the maximum posterior classifier is given in the following equation (3):

K-Nearest Neighbor (KNN)

One common example-based classifier is the K-nearest neighbor (KNN). Based on the similarity score, the search finds the K-nearest neighbors across all training reviews given a test review d [38]. The following equation (4) can be used to express the weighted sum in KNN categorization:

Support Vector Machine (SVM)

SVM is a powerful technique for solving problems in non-linear classification, function estimation and density estimation, which has led to many recent developments in kernel-based learning methods. Transforming a multi-label classification problem into a set of independent binary classification problems via the one-vs-all scheme is a conceptually simple and computationally ecient solution for multi- -label classification. In this work, we conduct multi-label learning under such a mechanism by using standard support vector machines (SVMs) for the binary classification problems associated with each class. Given a labelled multi- -label training set D={(xi,yi)}N i=1 where xi is the input feature vector for the i-th instance, and its label vector yi is a{+1,-1} valued vector with length K such as as K = |Y|. If Yik = 1, it indicates that the instance xi is assigned to the kth class; otherwise, the instance does not belong to the k-th class. For the k-th class (k = 1, · · ·, K) [4,36].

Experimental Setting

This section delineates the methodological approach employed to evaluate the effectiveness of text-based Arabic emotion detection and analysis models. Numerous experiments were conducted to assess both baseline and enhanced models. All experiments were carried out utilizing the SemEval-2018 dataset, which serves as the cornerstone of this research. The Arabic SemEval-2018 dataset is a corpus specifically curated for the SemEval (Semantic Evaluation) competition held in 2018. This dataset is sourced from Arabic text snippets extracted from Twitter, making it inherently rich in informal and colloquial language commonly found on social media platforms. The corpus is meticulously annotated with labels corresponding to eleven distinct emotions: anger, anticipation, disgust, fear, happiness, love, optimism, pessimism, sadness, surprise, and trust. Each text snippet within the dataset is categorized into one or more of these emotion categories based on the emotion expressed within the tweet. The dataset is partitioned in our experiments into two subsets: training, and testing, enabling researchers and practitioners to train, validate, and evaluate their models effectively. Table 1 provides an overview of the SemEval-2018 dataset.

To evaluate the proposed model, standard classifi- cation measurement precision, recall and F-measure are used Precision (Pi), Recall (Ri) and F-Measure (Fi) are mathematically defined shown in equations (5), (6) and (7).

Result and Discussion

Individual Supervised Model Experiments

Multiple experiments were conducted to gauge the performance of individual supervised machine learning models, coupled with the Synthetic Minority Oversampling Technique (SMOTE) to address data imbalance. Initially, a series of tests are conducted to assess three basic text-based Arabic emotion analysis models: Support Vector Machine (SVM), K-nearest neighbors (KNN) classifier, and Naive Bayes (NB). Figure 3 shows the performance (F-measure) of the top outcomes from the fundamental models of text-based Arabic emotion analysis. The SemEval-2018 dataset was used for all of the tests. The main objective of this research is to examine how well standard machine learning performs when it comes to emotion identification and analysis on both balanced and unbalanced data using SMOTE. One can see that the K Nearest Neighbor (KNN) classification approach yields less accurate results than the Support Vector Machine (SVM) methods. The results also show that traditional machine learning working on a balanced dataset by SMOTE outperforms traditional machine learning working on an unbalanced dataset. It can be observed that the meta-ensemble model outperforms other basic classifiers. The meta-classifier combines the strength of its individuals (basic classifiers). He is waiting for her when many individual classifiers agree on the majority classification cases and do not agree only for small cases (when one of them is wrong), the combination of these classifiers always gets higher scores. In addition, the combination of the decisions of several unique classifiers that take a high score is better than the individual classifier (base classifier).

Ensemble Model Experiments

This study conducted various experiments to assess the effectiveness of ensembles of supervised learning models, also utilizing the SMOTE technique to tackle data imbalance. several experiments are conducted to evaluate the proposed meta-classifier ensemble learning model which combines a set of supervised learning models for Arabic text-based emotion analysis. This meta-classifier ensemble learning combines NB, KNN and SVM. Table 3, and Figure 4 show the results of the proposed ensemble method. Comparing these results with the performances of other classifiers in an isolated method. including Support Vector Machine (SVM), K-nearest neighbor (KNN), and Naive Bayes (NB), it becomes evident that the meta-classifier ensemble consistently achieves competitive or superior performance, particularly when SMOTE is employed. Across different feature sizes, MCE with SMOTE consistently demonstrates higher Precision, Recall, and F-measure values compared to other classifiers with SMOTE. These findings highlight the effectiveness of MCE as a robust classification approach for imbalanced datasets, especially when combined with SMOTE to address class imbalance.

Overall, the best-performing classifier, both single and ensemble, was the meta classifier ensemble (MCE), achieving the highest accuracy of 87.51% on the balanced dataset. This highlights the effectiveness of ensemble learning techniques in improving emotion classification accuracy for Arabic text-based datasets. Moreover, the results emphasize the importance of addressing class imbalance in emotion classification tasks, with SMOTE proving to be a valuable technique for enhancing the performance of classification models on imbalanced datasets. These findings contribute to the advancement of emotion analysis in Arabic text.

Conclusion

This study conducts an empirical assessment of three foundational machine learning techniques: Support Vector Machine (SVM), K-Nearest Neighbors (KNN) classifier, and Naive Bayes (NB). Furthermore, it presents an Ensemble Framework specifically designed for Imbalanced Arabic Text-Based Emotion Analysis. The proposed approach demonstrates a commendable F-measure (F-score) of 87.51%, surpassing the performance of the basic methods. The results indicate that the proposed analytical method significantly enhances the detection and analysis of emotions in Arabic text. These outcomes suggest that the ensemble learning technique introduced is effective for the task of Text-based Arabic emotion detection and analysis. Future investigations should aim to develop emotion databases that encompass various Arabic dialects and slang, evaluate the proposed method across a range of datasets to confirm the model's efficacy and integrate advanced deep learning techniques to enhance ensemble learning strategies.