Classification of Imbalanced Offensive Dataset – Sentence Generation for Minority Class with LSTM

Ekin Ekinci

doi:10.35377/saucis...1070822

EN

Classification of Imbalanced Offensive Dataset – Sentence Generation for Minority Class with LSTM

Abstract

The classification of documents is one of the problems studied since ancient times and still continues to be studied. With the social media becoming a part of daily life and its misuse, the importance of text classification has started to increase. This paper investigates the effect of data augmentation with sentence generation on classification performance in an imbalanced dataset. We propose an LSTM based sentence generation method, Term Frequency-Inverse Document Frequency (TF-IDF) and Word2vec and apply Logistic Regression (LR), Support Vector Machine (SVM), K Nearest Neighbour (KNN), Multilayer Perceptron (MLP), Extremly Randomized Trees (Extra tree), Random Forest, eXtreme Gradient Boosting (Xgboost), Adaptive Boosting (AdaBoost) and Bagging. Our experiment results on imbalanced Offensive Language Identification Dataset (OLID) that machine learning with sentence generation significantly outperforms.

Keywords

References

[1] S. Rosenthal, P. Atanasova, G. Karadzhov, M. Zampieri, and P. Nakov, "OLID: A Large-Scale Semi-Supervised Dataset for Offensive Language Identification," arXiv preprint arXiv:2004.14454, 2020.
[2] G. Wiedemann, E. Ruppert, R. Jindal and C. Biemann, "Transfer Learning from LDA to BiLSTM-CNN for Offensive Language Detection in Twitter," arXiv preprint arXiv:1811.02906v1, 2018.
[3] H. Mubarak and K. Darwish K., "Arabic Offensive Language Classification on Twitter," Lecture Notes in Computer Science. Springer, Cham, 2019.
[4] E. Ekinci, S. İlhan Omurca and S. Sevim, "Improve Offensive Language Detection with Ensemble Classifiers," IJISAE, vol. 8, no. 2, pp. 109–115, 2020.
[5] M. Djandji, F. Baly, W. Antoun and H. Hajj, "Multi-Task Learning using AraBert for Offensive Language Detection," Proc. - 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pp. 97–101, 2020.
[6] Y. Tung and Y. Q. Zhang, "Granular SVM with Repetitive Undersampling for Highly Imbalanced Protein Homology Prediction," Proc. - 2006 IEEE International Conference on Granular Computing, pp. 457–460, 2006.
[7] J. Brownlee, Imbalanced Classification with Python. Machine Learning Mastery, 2020.
[8] Q. Zou, S. Xie, Z. Lin, M. Wu and Y. Ju, "Imbalanced classification is one of most popular topics in the field of machine learning," Big Data Res., vol. 5, pp. 2–8, 2016.

[9] L. Wang, H. Cheng, Z. Zheng, A. Yang and X. Zhu, " Ponzi scheme detection via oversampling-based Long Short-Term Memory for smart contracts," Knowl Based Syst., vol. 228, pp.1–12, 2021.
[10] A. Gosain and S. Sardana, "Handling Class Imbalance Problem using Oversampling Techniques: A Review," Proc. - 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 79–85, 2017.
[11] E. L. Iglesias, A. S. Vieira and L. Borrajo, " An HMM-based over-sampling technique to improve text classification," Expert Syst. Appl., 465, pp. 1–20, 2013.
[12] N. V. Chawla, K. W. Bowyer, L. O. Hall, W. P. Kegelmeyer, "SMOTE: Synthetic Minority Over-sampling Technique," J Artif Intell Res., vol. 16, pp. 321–357, 2002.
[13] H. A. Majzoub, I. Elgedawy, Ö. Akaydın and M. Köse Ulukök, " HCAB SMOTE: A Hybrid Clustered Afnitive Borderline SMOTE Approach for Imbalanced Data Binary Classifcation," Arab. J. Sci. Eng., vol. 45, pp. 3205–3222, 2020.
[14] G. Douzas, F. Bacao and F. Last, "Improving imbalanced learning through a heuristic over-sampling method based on k-means and SMOTE," Inf. Sci., vol. 465, pp. 1–20, 2018.
[15] C. Bunkhumpornpat, K. Sinapiromsaran and C. Lursinsap, "Safe-Level-SMOTE: Safe-Level-Synthetic Minority Over-Sampling TEchnique for Handling the Class Imbalanced Problem," Proc. - Pacific-Asia conference on knowledge discovery and data mining, pp. 475–482, 2009.
[16] S. Darabi and Y. Elor, "AE-SMOTE: A Multi-Modal Minority Oversampling Framework," pp. 1–19, 2020.
[17] A. Amin, S. Anwar, A. Adnan, M. Nawaz, N. Howard, J. Qadir, A. Hawalah and A. Hussain, "Comparing Oversampling Techniques to Handle the Class Imbalance Problem: A Customer Churn Prediction Case Study," IEEE Access, vol. 4, pp. 7940–7957, 2016.
[18] M. Zampieri, S. Malmasi, P. Nakov, S. Rosenthal, N. Farra and R. Kumar, " Predicting the Type and Target of Offensive Posts in Social Media," Proc. - NAACL-HLT, pp. 1415–1420, 2019.
[19] A. Rozental and D. Biton, "Amobee at SemEval-2019 Tasks 5 and 6: Multiple choice over contextual embedding," arXiv preprint arXiv:1904.08292., 2019.
[20] M. Sridharan and T. R. Swapna, "Amrita School of Engineering-CSE at SemEval-2019 Task 6: Manipulating attention with temporal convolutional neural network for offense identification and classification," Proc. - 13th International Workshop on SemEval, pp. 540–546, 2019.
[21] R. Kumar, G. Bhanodai, R. Pamula, and M. R. Chennuru, "bhanodaig at SemEval-2019 Task 6: Categorizing offensive language in social media," Proc. - 13th International Workshop on SemEval, pp. 547–550, 2019.
[22] Z. Wu, H. Zheng, J. Wang, W. Su and J. Fong, "Bnu-hkbu uic nlp team 2 at semeval-2019 task 6: Detecting offensive language using bert model," Proc. - 13th International Workshop on SemEval, pp. 551–555, 2019.
[23] G. Aglionby, C. Davis, P. Mishra, A. Caines, H. Yannakoudakis, M. Rei, E. Shutova and P. Buttery, "CAMsterdam at SemEval-2019 Task 6: Neural and graph-based feature extraction for the identification of offensive tweets," Proc. - 13th International Workshop on SemEval, pp. 556–563, 2019.
[24] Y. Zhang, B. Xu and T. Zhao, "CN-HIT-MI. T at SemEval-2019 Task 6: Offensive Language Identification Based on BiLSTM with Double Attention," Proc. - 13th International Workshop on SemEval, pp. 564–570, 2019.
[25] J. Pavlopoulos, N. Thain, L. Dixon and I. Androutsopoulos, "Convai at semeval-2019 task 6: Offensive language identification and categorization with perspective and bert," Proc. - 13th International Workshop on SemEval, pp. 571–576, 2019.
[26] S. Modha, P. Majumder, D. Patel, "DA-LD-Hildesheim at SemEval-2019 task 6: tracking offensive content with deep learning using shallow representation," Proc. - 13th International Workshop on SemEval, pp. 577–581, 2019.
[27] G. L. De la Peña and P. Rosso, "DeepAnalyzer at SemEval-2019 Task 6: A deep learning-based ensemble method for identifying offensive tweets," Proc. - 13th International Workshop on SemEval, pp. 582–586, 2019.
[28] T. Pedersen, "Duluth at SemEval-2019 task 6: Lexical approaches to identify and categorize offensive tweets," arXiv preprint arXiv:2007.12949, 2019.
[29] E. Kebriaei, S. Karimi, N. Sabri and A. Shakery, "Emad at SemEval-2019 task 6: offensive language identification using traditional machine learning and deep learning approaches," Proc. - 13th International Workshop on SemEval, pp. 600–603, 2019.
[30] A. Pelicon, M. Martinc and P. K. Novak, "Embeddia at semeval-2019 task 6: Detecting hate with neural network and transfer learning approaches," Proc. - 13th International Workshop on SemEval, pp. 604–610, 2019.
[31] V. Indurthi, B. Syed, M. Shrivastava, M. Gupta and V. Varma, "Fermi at SemEval-2019 Task 6: Identifying and categorizing offensive language in social media using sentence embeddings," Proc. - 13th International Workshop on SemEval, pp. 611–616, 2019.
[32] H. Bansal, D. Nagel and A. Soloveva, "HAD-Tübingen at SemEval-2019 Task 6: Deep learning analysis of offensive language on Twitter: Identification and categorization," Proc. - 13th International Workshop on SemEval, pp. 622–627, 2019.
[33] A. Oberstrass, J. Romberg, A. Stoll and S. Conrad, "HHU at SemEval-2019 Task 6: Context does matter-tackling offensive language identification and categorization with ELMo," Proc. - 13th International Workshop on SemEval, pp. 628–634, 2019.
[34] G. F. Patras, D. F. Lungu, D. Gifu and D. Trandabat, "Hope at SemEval-2019 Task 6: Mining social media language to discover offensive language," Proc. - 13th International Workshop on SemEval, pp. 635–638, 2019.
[35] M. Graff, S. Miranda-Jiménez, E. Tellez and D. A. Ochoa, "INGEOTEC at SemEval-2019 task 5 and task 6: A genetic programming approach for text classification," Proc. - 13th International Workshop on SemEval, pp. 639–644, 2019.
[36] Y. HaCohen-Kerner, Z. Ben-David, G. Didi, E. Cahn, S. Rochman and E. Shayovitz, "JCTICOL at SemEval-2019 Task 6: Classifying offensive language in social media using deep learning methods, word/character n-gram features, and preprocessing methods," Proc. - 13th International Workshop on SemEval, pp. 645–651, 2019.
[37] P. Mukherjee, M. Pal, S. Banerjee and S. K. Naskar, "JU_ETCE_17_21 at SemEval-2019 Task 6: Efficient Machine Learning and Neural Network Approaches for Identifying and Categorizing Offensive Language in Tweets," Proc. - 13th International Workshop on SemEval, pp. 662–667, 2019.
[38] P. Rani and A. K. Ojha, "KMI-coling at SemEval-2019 task 6: exploring N-grams for offensive language detection," Proc. - 13th International Workshop on SemEval, pp. 668–671, 2019.
[39] L. S. M. Altın, À. B. Serrano and H. Saggion, "Lastus/taln at semeval-2019 task 6: Identification and categorization of offensive language in social media with attention-based bi-lstm model," Proc. - 13th International Workshop on SemEval, pp. 672–677, 2019.
[40] P. Aggarwal, T. Horsmann, M. Wojatzki and T. Zesch, "LTL-UDE at SemEval-2019 Task 6: BERT and two-vote classification for categorizing offensiveness," Proc. - 13th International Workshop on SemEval, pp. 678–682, 2019.
[41] E. Doostmohammadi, H. Sameti and A. Saffar, "Ghmerti at SemEval-2019 task 6: a deep word-and character-based approach to offensive language identification," arXiv preprint arXiv:2009.10792, 2020.
[42] N. Oswal, "SemEval-2019 (OffensEval): Identifying and Categorizing Offensive Language in Social Media," arXiv preprint arXiv: 2104.04871v1, 2021.
[43] D. Sarkar, M. Zampieri, T. Ranasinghe and A. Orarbia, "fBERT: A Neural Transformer for Identifying Offensive Content," arXiv preprint arXiv: 2109.05074v1, 2021.
[44] F. Muslim, A. Purwarianti and F. Z. Ruskanda, "Cost-Sensitive Learning and Ensemble BERT for Identifying and Categorizing Offensive Language in Social Media," Proc. - ICAICTA, pp. 1–6, 2021.
[45] A. S. Neogi, K. A. Garg, R. K. Mishra and Y. K. Dwivedi, "Sentiment analysis and classification of Indian farmers’ protest using twitter data," Int. J. Inf. Manage., vol. 1, no. 2, pp. 100019, 2021.
[46] E. M. Dharma, F. L. Gaol, H. L. H. S. Warnars and B. Soewito, "The Accuracy Comparison Among Word2vec, Glove, And Fasttext Towards Convolution Neural Network (CNN) Text Classification," J. Theor. Appl. Inf., vol. 100, no. 2, pp. 349–359, 2022.
[47] M. S. Başarslan and F. Kayaalp, " Sentiment Analysis on Social Media Reviews Datasets with Deep Learning Approach," SAUCIS, vol. 4, no. 1, pp. 35–49, 2021.
[48] J. V. Lochter, P. R. Pires, C. Bossolani, A. Yamakami and T. A. Almeida, " Evaluating the impact of corpora used to train distributed text representation models for noisy and short texts," Proc. - 2018 International Joint Conference on Neural Networks, pp. 1–8, 2018.
[49] A. Zhao, L. Qi, J. Dong and H. Yu, "Dual channel LSTM based multi-feature extraction in gait for diagnosis of Neurodegenerative diseases," Knowl. Based Syst., vol. 145, pp. 91–97, 2018.
[50] B. Kaya and A. Günay, "Twitter Sentiment Analysis Based on Daily Covid-19 Table in Turkey," SAUCIS, vol. 4, no. 3, pp. 302–311, 2021.
[51] Y. Canbay, A. İsmetoğlu and P. Canbay, " Deep Learning and Data Privacy in Diagnosis of Covid-19," J. Eng. Sci. Technol., vol. 9, no. 2, pp. 701–715, 2021.
[52] E. Ekinci, S. İlhan Omurca and N. Acun, "A Comparative Study on Machine Learning Techniques using Titanic Dataset," Proc. - 7th International Conference on Advanced Technologies, pp. 411–416, 2018.
[53] D. Chen, H. Bourlard and J. P. Thiran, "Text identification in complex background using SVM," Proc. - 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 621–626, 2001.
[54] M. Jogin, M. S. Madhulika, G. D. Divya, R. K. Meghana, and S. Apoorva, "Feature Extraction using Convolution Neural Networks (CNN) and Deep Learning," Proc. - 3rd IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology, pp. 2319–2323, 2018.
[55] S. Lallahem, J. Mania, A. Hani and Y. Najjar, "On the use of neural networks to evaluate groundwater levels in fractured media," J. Hydrol., vol. 307, no. 1-4, pp. 92–111, 2005.
[56] K. Kaur and S. K. Mittal, "Classification of mammography image with CNN-RNN based semantic features and extra tree classifier approach using LSTM," Mater. Today., pp. 1–7, 2020.
[57] S. Sevim, E. Ekinci and S. İlhan Omurca, "Multi-view Document Classification with Co-training," Proc. - 28th IEEE Conference on Signal Processing and Communications Applications, pp. 1–4, 2020.
[58] D. A. Rusdah and H. Murfi, "XGBoost in handling missing values for life insurance risk prediction," SN Appl. Sci., vol. 2, no. 8, pp. 1–10, 2020.
[59] T. Chen and C. Guestrin, "XGBoost: A Scalable Tree Boosting System," arXiv preprint arXiv: 1603.02754v3.
[60] E. Ekinci and H. Takçı, "Comparing ensemble classifiers: Forensic analysis of electronic mails," Global Journal on Technology, vol. 4, no. 2, pp. 167–173, 2013.
[61] G. Liang, X. Zhu, and C. Zhang, "An empirical stıdy of bagging predictors for different learning algorithms," Proc. - AAAI'11, pp. 1802–1803, 2011.

Details

Primary Language

English

Subjects

Artificial Intelligence

Journal Section

Research Article

Authors

Ekin Ekinci ^*
0000-0003-0658-592X
Türkiye

Publication Date

April 30, 2022

Submission Date

February 10, 2022

Acceptance Date

April 18, 2022

Published in Issue

Year 1970 Volume: 5 Number: 1

DOI

https://doi.org/10.35377/saucis...1070822

IZ

https://izlik.org/JA29AW52AH

Cite

RIS / Bibtex

APA

Ekinci, E. (2022). Classification of Imbalanced Offensive Dataset – Sentence Generation for Minority Class with LSTM. Sakarya University Journal of Computer and Information Sciences, 5(1), 121-133. https://doi.org/10.35377/saucis...1070822

AMA

1.Ekinci E. Classification of Imbalanced Offensive Dataset – Sentence Generation for Minority Class with LSTM. SAUCIS. 2022;5(1):121-133. doi:10.35377/saucis.1070822

Chicago

Ekinci, Ekin. 2022. “Classification of Imbalanced Offensive Dataset – Sentence Generation for Minority Class With LSTM”. Sakarya University Journal of Computer and Information Sciences 5 (1): 121-33. https://doi.org/10.35377/saucis. 1070822.

EndNote

Ekinci E (April 1, 2022) Classification of Imbalanced Offensive Dataset – Sentence Generation for Minority Class with LSTM. Sakarya University Journal of Computer and Information Sciences 5 1 121–133.

IEEE

[1]E. Ekinci, “Classification of Imbalanced Offensive Dataset – Sentence Generation for Minority Class with LSTM”, SAUCIS, vol. 5, no. 1, pp. 121–133, Apr. 2022, doi: 10.35377/saucis...1070822.

ISNAD

Ekinci, Ekin. “Classification of Imbalanced Offensive Dataset – Sentence Generation for Minority Class With LSTM”. Sakarya University Journal of Computer and Information Sciences 5/1 (April 1, 2022): 121-133. https://doi.org/10.35377/saucis. 1070822.

JAMA

1.Ekinci E. Classification of Imbalanced Offensive Dataset – Sentence Generation for Minority Class with LSTM. SAUCIS. 2022;5:121–133.

MLA

Ekinci, Ekin. “Classification of Imbalanced Offensive Dataset – Sentence Generation for Minority Class With LSTM”. Sakarya University Journal of Computer and Information Sciences, vol. 5, no. 1, Apr. 2022, pp. 121-33, doi:10.35377/saucis. 1070822.

Vancouver

1.Ekin Ekinci. Classification of Imbalanced Offensive Dataset – Sentence Generation for Minority Class with LSTM. SAUCIS. 2022 Apr. 1;5(1):121-33. doi:10.35377/saucis. 1070822

Classification of Imbalanced Offensive Dataset – Sentence Generation for Minority Class with LSTM

Classification of Imbalanced Offensive Dataset – Sentence Generation for Minority Class with LSTM

Abstract

Keywords

Öz

References

Details

Primary Language

Subjects

Journal Section

Authors

Publication Date

Submission Date

Acceptance Date

Published in Issue

DOI

IZ

Cite

Cited By

Derin ve Sığ Makine Öğrenmesi Yöntemleri ile Türkçe Tweetlerden Saldırgan Dil Tespiti

Self Adaptive Methods for Learning Rate Parameter of Q-Learning Algorithm

Optimizing Soil Fertility through Machine Learning: Enhancing Agricultural Productivity and Sustainability