Sentiment Analysis on Social Media Reviews Datasets with Deep Learning Approach

Muhammet Sinan Başarslan; Fatih Kayaalp

doi:10.35377/saucis.04.01.833026

EN

Sentiment Analysis on Social Media Reviews Datasets with Deep Learning Approach

Abstract

Thanks to social media, people are now able to leave guiding comments quickly about their favorite restaurants, movies, etc. This has paved the way for the field of sentiment analysis, which brings together various disciplines. In this study, Yelp restaurant reviews and IMDB movie reviews dataset were used together with the data collected from Twitter. Word2Vec (W2V), Global Vector (GloVe) and Bidirectional Encoder Representation (BERT) word embedding methods, Term Frequency-Reverse Document Frequency (TF-IDF), and the Bag-of-Words (BOW) were used on these datasets. Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Recurrent Neural Network (RNN), Support Vector Machine (SVM), and Naive Bayes (NB) were used in the sentiment analysis models. Accuracy, F-measure (F), Sensitivity (Sens), Precision (Pre), and Receiver Operating Characteristics (ROC) were used in the evaluation of the model performance. The Accuracy rates of the models created by the Machine Learning (ML) and Deep Learning (DL) methods using the IMDB dataset were in the range of 81%-90% and 84%-94%, respectively. These rates were in the range of 80%-86% and 81%-89% for the Yelp dataset, and in the range of 75%-79% and 85%-98% for the Twitter dataset. The models that incorporated the BERT word embedding method have the best performance, compared to the other models with ML and DL. Therefore, BERT method is recommended for this type of analysis in future studies.

Keywords

References

E. Park, J. Kang, D. Choi, and J. Han, “Understanding Customers' Hotel Revisiting Behaviour: a sentiment analysis of Online Feedback Reviews,” Current Issues in Tourism, vol. 23, pp. 605-611, 2020, doi: 10.1080/13683500.2018.1549025.
B. Pang and L. Lee, "Opinion mining and sentiment analysis", Foundations Trends Information Retrival, vol. 2, no. 2, 2008, pp. 1-135.
O. Kaynar, H. Arslan, Y. Görmez and F. Demirkoparan, "Makine Öğrenmesi Yöntemleri ile Duygu Analizi," International Artificial Intelligence and Data Processing Symposium (IDAP), pp. 1-5, Malatya, 2017.
A. Al Hamoud, A. Alwehaibi, K. Roy, and M. Bikdash, “Classifying Political Tweets Using Naïve Bayes and Support Vector Machines,” In International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, pp. 736-744, 2018, doi: 10.1007/978-3-319-92058-0_71.
S. Symeonidis, D. Effrosynidis, and A, Arampatzis, “A Comparative Evaluation of Pre‐Processing Techniques and Their Interactions for Twitter Sentiment Analysis,” Expert System Applications, vol. 110, pp. 298-310, 2018, doi: 10.1016/j.eswa.2018.06.022.
M. A. Paredes-Valverde, R. Colomo-Palacios, M. P. Salas-Zárate, and R. Valencia-García, “Sentiment Analysis in Spanish for Improvement of Products and Services: A Deep Learning Approach,” Scientific Programming, vol. 2017, 2017, doi: 10.1155/2017/1329281.
J. Zheng and L. Zheng, "A Hybrid Bidirectional Recurrent Convolutional Neural Network Attention-Based Model for Text Classification," IEEE Access, vol. 7, 2019, pp. 106673-106685, doi: 10.1109/ACCESS.2019.2932619.
S. Liu, “Sentiment Analysis of Yelp Reviews: A Comparison of Techniques and Models”, arXiv preprint, arXiv:2004.13851, 2020.

M. R. Huq, A. Ali, and A. Rahman, “Sentiment Analysis on Twitter Data Using KNN and SVM,” International Journal of Advanced Computer Science and Applications, vol. 8, pp. 19-25, 2017, doi: 10.14569/IJACSA.2017.080603.
A. Amolik, N. Jivane, M. Bhandari, and M. Venkatesan “Twitter Sentiment Analysis of Movie Reviews Using Machine Learning Techniques,” International Journal of Engineering and Technology, vol. 7, no. 6, pp. 1-7, 2016.
S. Liao J. Wang R. Yu, K. Sato, and Z., Cheng, “CNN for Situations Understanding Based on Sentiment Analysis of Twitter Data,” Procedia Computer Science, vol. 111, 2017, pp. 376–381, 2017, doi: 10.1016/j.procs.2017.06.037
Li C, Guo X, Mei Q (2017b) Deep Memory Networks for Attitude Identification. In: Proceedings of the tenth ACM International Conference on Web Search and Data Mining, WSDM, Cambridge, United Kingdom, pp 671–680, 2017.
B. Li, Z. Cheng, Z. Xu, W. Ye, T. Lukasiewicz and S. Zhang, “Long Text Analysis Using Sliced Recurrent Neural Networks with Breaking Point Information Enrichment,” ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, United Kingdom, pp. 7550-7554, 2019,doi: 10.1109/ICASSP.2019.8683812.
W. Zhao et al., "Weakly-Supervised Deep Embedding for Product Review Sentiment Analysis," IEEE Transactions on Knowledge and Data Engineering, vol. 30, no. 1, 1 Jan. pp. 185-197, 2018, doi: 10.1109/TKDE.2017.2756658.
M. Al-Smadi, O. Qawasmeh, M. Al-Ayyoub, Y. Jararweh, and B. Gupta, “Deep Recurrent Neural Network vs. Support Vector Machine for Aspect-Based Sentiment Analysis of Arabic Hotels’ Reviews,” Journal of Computational Science, 2017, doi: 10.1016/j.jocs.2017.11.006.
D. Tang, F. Wei, B. Qin, N. Yang, T. Liu, and M. Zhou, “Sentiment Embeddings with Applications to Sentiment Analysis,” In IEEE Transactions on Knowledge and Data Engineering: vol. 28, pp. 496–509, 2016,doi: 10.1109/TKDE.2015.2489653.
P. Chen, Z. Sun, L. Bing, and W. Yang, “Recurrent Attention Network on Memory for Aspect Sentiment Analysis,” Empirical Methods in Natural Language Processing, pp. 452–461, 2017.
F. Tian et al., “Recognizing and Regulating Elearners’ Emotions Based on interactive Chinese Texts in E-Learning Systems,” Knowledge Based System, vol. 55, 148–164, 2014, doi: 10.1016/j.knosys.2013.10.019
H. Ghulam, F. Zeng, W. Li, and Y. Xiao, "Deep learning-based Sentiment Analysis for Roman Urdu Text," Procedia Computer Science, vol. 147, pp.131-135, 2019, doi: 10.1016/j.procs.2019.01.202
J. Singh, R. Singh, and P. Singh, "Morphological evaluation and sentiment analysis of Punjabi text using deep learning classification," Journal King Saud University-Computer and Information Science, 2018, doi: 10.1016/j.jksuci.2018.04.003.
Yelp Polarity Dataset, “TensorFlow Datasets Catalog homepage,” 2015. [online]. Available: https://www.tensorflow.org/datasets/catalog/yelp_polarity_reviews
A. L. Maas, R.E. Daly, P.T. Pham, D. Huang, A.Y. Ng and C. Potts, "Learning Word Vectors for Sentiment Analysis", Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 142-150, 2011.
R. Sjögren, K. Stridh, T. Skotare, J. and J. Trygg, "Multivariate Patent Analysis-Using Chemometrics to Analyze Collections of Chemical and Pharmaceutical Patents," Journal of Chemometrics, vol. 34, pp. e3041, 2020, doi: 10.1002/cem.3041
A. Onan "Mining opinions from instructor evaluation reviews: A Deep Learning Approach, " Computer Application in Engineering Education, vol. 28, pp. 117–138, 2020, doi: 10.1002/cae.22179.
T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space”, arXiv preprint, arXiv:1301.3781, 2013.
T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean, "Distributed Representations of Words and Phrases and Their Compositionality," Neural Information Processing Systems Conference, Lake Tahoe, pp. 3111–3119, 2013.
R. Ni and H. Cao, "Sentiment Analysis based on GloVe and LSTM-GRU," 39th Chinese Control Conference (CCC), Shenyang, China, pp. 7492-7497, 2020, doi: 10.23919/CCC50068.2020.9188578.
M. M. Saritas, A. Yasar, "Performance Analysis of ANN and Naive Bayes Classification Algorithm for Data Classification," International Journal of Intelligent Systems and Applications in Engineering, vol. 7, pp. 88-91,2019, doi: 10.18201//ijisae.2019252786.
S. Qing, H. Wenjie and X. Wenfang, "Robust Support Vector Machine with Bullet Hole Image Classification," IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 32, no. 4, pp. 440-448, 2002, doi: 10.1109/TSMCC.2002.807277.
I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA, USA: MIT Press, 2016.
S. Karita et al., "A Comparative Study on Transformer vs RNN in Speech Applications," IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), SG, Singapore, , pp. 449-456, 2019, doi: 10.1109/ASRU46091.2019.9003750.
L. M. Rojas-Barahona, "Deep Learning for Sentiment Analysis," Language Linguistic Compass, vol. 10, no. 12, 2016, doi: 10.1111/lnc3.12228
Y. LeCun, Y. Bengio, ang G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, 2015, doi: 10.1038/nature14539.
Ş. Kayıkçı,“A convolutional neural network model implementation for speech recognition,” Düzce Üniversitesi Bilim ve Teknoloji Dergisi, vol. 7, no. 3, pp. 1892-1898, 2019, doi: 10.29130/dubited.567828.
M. S. Başarslan and F. Kayaalp, "Performance Analysis Of Fuzzy Rough Set-Based And Correlation-Based Attribute Selection Methods On Detection Of Chronic Kidney Disease With Various Classifiers," 2019 Scientific Meeting on Electrical-Electronics & Biomedical Engineering and Computer Science (EBBT), Istanbul, Turkey, 2019, pp. 1-5. doi: 10.1109/EBBT.2019.8741688.
K. Polat, and S. Güneş, “Breast cancer diagnosis using least square support vector machine,” Digital signal processing, vol. 17, no. 4, pp. 694-701, 2007, doi: 10.1016/j.dsp.2006.10.008.

Details

Primary Language

English

Subjects

Artificial Intelligence

Journal Section

Research Article

Authors

Muhammet Sinan Başarslan ^*
0000-0002-7996-9169
Türkiye

Fatih Kayaalp
0000-0002-8752-3335
Türkiye

Publication Date

April 30, 2021

Submission Date

November 28, 2020

Acceptance Date

February 4, 2021

Published in Issue

Year 1970 Volume: 4 Number: 1

DOI

https://doi.org/10.35377/saucis.04.01.833026

IZ

https://izlik.org/JA59JW89YR

Cite

RIS / Bibtex

APA

Başarslan, M. S., & Kayaalp, F. (2021). Sentiment Analysis on Social Media Reviews Datasets with Deep Learning Approach. Sakarya University Journal of Computer and Information Sciences, 4(1), 35-49. https://doi.org/10.35377/saucis.04.01.833026

AMA

1.Başarslan MS, Kayaalp F. Sentiment Analysis on Social Media Reviews Datasets with Deep Learning Approach. SAUCIS. 2021;4(1):35-49. doi:10.35377/saucis.04.01.833026

Chicago

Başarslan, Muhammet Sinan, and Fatih Kayaalp. 2021. “Sentiment Analysis on Social Media Reviews Datasets With Deep Learning Approach”. Sakarya University Journal of Computer and Information Sciences 4 (1): 35-49. https://doi.org/10.35377/saucis.04.01.833026.

EndNote

Başarslan MS, Kayaalp F (April 1, 2021) Sentiment Analysis on Social Media Reviews Datasets with Deep Learning Approach. Sakarya University Journal of Computer and Information Sciences 4 1 35–49.

IEEE

[1]M. S. Başarslan and F. Kayaalp, “Sentiment Analysis on Social Media Reviews Datasets with Deep Learning Approach”, SAUCIS, vol. 4, no. 1, pp. 35–49, Apr. 2021, doi: 10.35377/saucis.04.01.833026.

ISNAD

Başarslan, Muhammet Sinan - Kayaalp, Fatih. “Sentiment Analysis on Social Media Reviews Datasets With Deep Learning Approach”. Sakarya University Journal of Computer and Information Sciences 4/1 (April 1, 2021): 35-49. https://doi.org/10.35377/saucis.04.01.833026.

JAMA

1.Başarslan MS, Kayaalp F. Sentiment Analysis on Social Media Reviews Datasets with Deep Learning Approach. SAUCIS. 2021;4:35–49.

MLA

Başarslan, Muhammet Sinan, and Fatih Kayaalp. “Sentiment Analysis on Social Media Reviews Datasets With Deep Learning Approach”. Sakarya University Journal of Computer and Information Sciences, vol. 4, no. 1, Apr. 2021, pp. 35-49, doi:10.35377/saucis.04.01.833026.

Vancouver

1.Muhammet Sinan Başarslan, Fatih Kayaalp. Sentiment Analysis on Social Media Reviews Datasets with Deep Learning Approach. SAUCIS. 2021 Apr. 1;4(1):35-49. doi:10.35377/saucis.04.01.833026