Sentiment Analysis on Social Media Reviews Datasets with Deep Learning Approach

Muhammet Sinan Başarslan; Fatih Kayaalp

doi:10.35377/saucis.04.01.833026

Research Article

Year 2021, Volume: 4 Issue: 1, 35 - 49, 30.04.2021

Muhammet Sinan Başarslan , Fatih Kayaalp

https://doi.org/10.35377/saucis.04.01.833026

Cited By: 13

Abstract

References

E. Park, J. Kang, D. Choi, and J. Han, “Understanding Customers' Hotel Revisiting Behaviour: a sentiment analysis of Online Feedback Reviews,” Current Issues in Tourism, vol. 23, pp. 605-611, 2020, doi: 10.1080/13683500.2018.1549025.
B. Pang and L. Lee, "Opinion mining and sentiment analysis", Foundations Trends Information Retrival, vol. 2, no. 2, 2008, pp. 1-135.
O. Kaynar, H. Arslan, Y. Görmez and F. Demirkoparan, "Makine Öğrenmesi Yöntemleri ile Duygu Analizi," International Artificial Intelligence and Data Processing Symposium (IDAP), pp. 1-5, Malatya, 2017.
A. Al Hamoud, A. Alwehaibi, K. Roy, and M. Bikdash, “Classifying Political Tweets Using Naïve Bayes and Support Vector Machines,” In International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, pp. 736-744, 2018, doi: 10.1007/978-3-319-92058-0_71.
S. Symeonidis, D. Effrosynidis, and A, Arampatzis, “A Comparative Evaluation of Pre‐Processing Techniques and Their Interactions for Twitter Sentiment Analysis,” Expert System Applications, vol. 110, pp. 298-310, 2018, doi: 10.1016/j.eswa.2018.06.022.
M. A. Paredes-Valverde, R. Colomo-Palacios, M. P. Salas-Zárate, and R. Valencia-García, “Sentiment Analysis in Spanish for Improvement of Products and Services: A Deep Learning Approach,” Scientific Programming, vol. 2017, 2017, doi: 10.1155/2017/1329281.
J. Zheng and L. Zheng, "A Hybrid Bidirectional Recurrent Convolutional Neural Network Attention-Based Model for Text Classification," IEEE Access, vol. 7, 2019, pp. 106673-106685, doi: 10.1109/ACCESS.2019.2932619.
S. Liu, “Sentiment Analysis of Yelp Reviews: A Comparison of Techniques and Models”, arXiv preprint, arXiv:2004.13851, 2020.
M. R. Huq, A. Ali, and A. Rahman, “Sentiment Analysis on Twitter Data Using KNN and SVM,” International Journal of Advanced Computer Science and Applications, vol. 8, pp. 19-25, 2017, doi: 10.14569/IJACSA.2017.080603.
A. Amolik, N. Jivane, M. Bhandari, and M. Venkatesan “Twitter Sentiment Analysis of Movie Reviews Using Machine Learning Techniques,” International Journal of Engineering and Technology, vol. 7, no. 6, pp. 1-7, 2016.
S. Liao J. Wang R. Yu, K. Sato, and Z., Cheng, “CNN for Situations Understanding Based on Sentiment Analysis of Twitter Data,” Procedia Computer Science, vol. 111, 2017, pp. 376–381, 2017, doi: 10.1016/j.procs.2017.06.037
Li C, Guo X, Mei Q (2017b) Deep Memory Networks for Attitude Identification. In: Proceedings of the tenth ACM International Conference on Web Search and Data Mining, WSDM, Cambridge, United Kingdom, pp 671–680, 2017.
B. Li, Z. Cheng, Z. Xu, W. Ye, T. Lukasiewicz and S. Zhang, “Long Text Analysis Using Sliced Recurrent Neural Networks with Breaking Point Information Enrichment,” ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, United Kingdom, pp. 7550-7554, 2019,doi: 10.1109/ICASSP.2019.8683812.
W. Zhao et al., "Weakly-Supervised Deep Embedding for Product Review Sentiment Analysis," IEEE Transactions on Knowledge and Data Engineering, vol. 30, no. 1, 1 Jan. pp. 185-197, 2018, doi: 10.1109/TKDE.2017.2756658.
M. Al-Smadi, O. Qawasmeh, M. Al-Ayyoub, Y. Jararweh, and B. Gupta, “Deep Recurrent Neural Network vs. Support Vector Machine for Aspect-Based Sentiment Analysis of Arabic Hotels’ Reviews,” Journal of Computational Science, 2017, doi: 10.1016/j.jocs.2017.11.006.
D. Tang, F. Wei, B. Qin, N. Yang, T. Liu, and M. Zhou, “Sentiment Embeddings with Applications to Sentiment Analysis,” In IEEE Transactions on Knowledge and Data Engineering: vol. 28, pp. 496–509, 2016,doi: 10.1109/TKDE.2015.2489653.
P. Chen, Z. Sun, L. Bing, and W. Yang, “Recurrent Attention Network on Memory for Aspect Sentiment Analysis,” Empirical Methods in Natural Language Processing, pp. 452–461, 2017.
F. Tian et al., “Recognizing and Regulating Elearners’ Emotions Based on interactive Chinese Texts in E-Learning Systems,” Knowledge Based System, vol. 55, 148–164, 2014, doi: 10.1016/j.knosys.2013.10.019
H. Ghulam, F. Zeng, W. Li, and Y. Xiao, "Deep learning-based Sentiment Analysis for Roman Urdu Text," Procedia Computer Science, vol. 147, pp.131-135, 2019, doi: 10.1016/j.procs.2019.01.202
J. Singh, R. Singh, and P. Singh, "Morphological evaluation and sentiment analysis of Punjabi text using deep learning classification," Journal King Saud University-Computer and Information Science, 2018, doi: 10.1016/j.jksuci.2018.04.003.
Yelp Polarity Dataset, “TensorFlow Datasets Catalog homepage,” 2015. [online]. Available: https://www.tensorflow.org/datasets/catalog/yelp_polarity_reviews
A. L. Maas, R.E. Daly, P.T. Pham, D. Huang, A.Y. Ng and C. Potts, "Learning Word Vectors for Sentiment Analysis", Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 142-150, 2011.
R. Sjögren, K. Stridh, T. Skotare, J. and J. Trygg, "Multivariate Patent Analysis-Using Chemometrics to Analyze Collections of Chemical and Pharmaceutical Patents," Journal of Chemometrics, vol. 34, pp. e3041, 2020, doi: 10.1002/cem.3041
A. Onan "Mining opinions from instructor evaluation reviews: A Deep Learning Approach, " Computer Application in Engineering Education, vol. 28, pp. 117–138, 2020, doi: 10.1002/cae.22179.
T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space”, arXiv preprint, arXiv:1301.3781, 2013.
T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean, "Distributed Representations of Words and Phrases and Their Compositionality," Neural Information Processing Systems Conference, Lake Tahoe, pp. 3111–3119, 2013.
R. Ni and H. Cao, "Sentiment Analysis based on GloVe and LSTM-GRU," 39th Chinese Control Conference (CCC), Shenyang, China, pp. 7492-7497, 2020, doi: 10.23919/CCC50068.2020.9188578.
M. M. Saritas, A. Yasar, "Performance Analysis of ANN and Naive Bayes Classification Algorithm for Data Classification," International Journal of Intelligent Systems and Applications in Engineering, vol. 7, pp. 88-91,2019, doi: 10.18201//ijisae.2019252786.
S. Qing, H. Wenjie and X. Wenfang, "Robust Support Vector Machine with Bullet Hole Image Classification," IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 32, no. 4, pp. 440-448, 2002, doi: 10.1109/TSMCC.2002.807277.
I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA, USA: MIT Press, 2016.
S. Karita et al., "A Comparative Study on Transformer vs RNN in Speech Applications," IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), SG, Singapore, , pp. 449-456, 2019, doi: 10.1109/ASRU46091.2019.9003750.
L. M. Rojas-Barahona, "Deep Learning for Sentiment Analysis," Language Linguistic Compass, vol. 10, no. 12, 2016, doi: 10.1111/lnc3.12228
Y. LeCun, Y. Bengio, ang G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, 2015, doi: 10.1038/nature14539.
Ş. Kayıkçı,“A convolutional neural network model implementation for speech recognition,” Düzce Üniversitesi Bilim ve Teknoloji Dergisi, vol. 7, no. 3, pp. 1892-1898, 2019, doi: 10.29130/dubited.567828.
M. S. Başarslan and F. Kayaalp, "Performance Analysis Of Fuzzy Rough Set-Based And Correlation-Based Attribute Selection Methods On Detection Of Chronic Kidney Disease With Various Classifiers," 2019 Scientific Meeting on Electrical-Electronics & Biomedical Engineering and Computer Science (EBBT), Istanbul, Turkey, 2019, pp. 1-5. doi: 10.1109/EBBT.2019.8741688.
K. Polat, and S. Güneş, “Breast cancer diagnosis using least square support vector machine,” Digital signal processing, vol. 17, no. 4, pp. 694-701, 2007, doi: 10.1016/j.dsp.2006.10.008.

Sentiment Analysis on Social Media Reviews Datasets with Deep Learning Approach

Year 2021, Volume: 4 Issue: 1, 35 - 49, 30.04.2021

Muhammet Sinan Başarslan , Fatih Kayaalp

https://doi.org/10.35377/saucis.04.01.833026

Cited By: 13

Abstract

Thanks to social media, people are now able to leave guiding comments quickly about their favorite restaurants, movies, etc. This has paved the way for the field of sentiment analysis, which brings together various disciplines. In this study, Yelp restaurant reviews and IMDB movie reviews dataset were used together with the data collected from Twitter. Word2Vec (W2V), Global Vector (GloVe) and Bidirectional Encoder Representation (BERT) word embedding methods, Term Frequency-Reverse Document Frequency (TF-IDF), and the Bag-of-Words (BOW) were used on these datasets. Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Recurrent Neural Network (RNN), Support Vector Machine (SVM), and Naive Bayes (NB) were used in the sentiment analysis models. Accuracy, F-measure (F), Sensitivity (Sens), Precision (Pre), and Receiver Operating Characteristics (ROC) were used in the evaluation of the model performance. The Accuracy rates of the models created by the Machine Learning (ML) and Deep Learning (DL) methods using the IMDB dataset were in the range of 81%-90% and 84%-94%, respectively. These rates were in the range of 80%-86% and 81%-89% for the Yelp dataset, and in the range of 75%-79% and 85%-98% for the Twitter dataset. The models that incorporated the BERT word embedding method have the best performance, compared to the other models with ML and DL. Therefore, BERT method is recommended for this type of analysis in future studies.

Keywords

sentiment analysis, Deep Learning, Machine Learning, Text Representation, Word Embedding

References

E. Park, J. Kang, D. Choi, and J. Han, “Understanding Customers' Hotel Revisiting Behaviour: a sentiment analysis of Online Feedback Reviews,” Current Issues in Tourism, vol. 23, pp. 605-611, 2020, doi: 10.1080/13683500.2018.1549025.
B. Pang and L. Lee, "Opinion mining and sentiment analysis", Foundations Trends Information Retrival, vol. 2, no. 2, 2008, pp. 1-135.
O. Kaynar, H. Arslan, Y. Görmez and F. Demirkoparan, "Makine Öğrenmesi Yöntemleri ile Duygu Analizi," International Artificial Intelligence and Data Processing Symposium (IDAP), pp. 1-5, Malatya, 2017.
A. Al Hamoud, A. Alwehaibi, K. Roy, and M. Bikdash, “Classifying Political Tweets Using Naïve Bayes and Support Vector Machines,” In International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, pp. 736-744, 2018, doi: 10.1007/978-3-319-92058-0_71.
S. Symeonidis, D. Effrosynidis, and A, Arampatzis, “A Comparative Evaluation of Pre‐Processing Techniques and Their Interactions for Twitter Sentiment Analysis,” Expert System Applications, vol. 110, pp. 298-310, 2018, doi: 10.1016/j.eswa.2018.06.022.
M. A. Paredes-Valverde, R. Colomo-Palacios, M. P. Salas-Zárate, and R. Valencia-García, “Sentiment Analysis in Spanish for Improvement of Products and Services: A Deep Learning Approach,” Scientific Programming, vol. 2017, 2017, doi: 10.1155/2017/1329281.
J. Zheng and L. Zheng, "A Hybrid Bidirectional Recurrent Convolutional Neural Network Attention-Based Model for Text Classification," IEEE Access, vol. 7, 2019, pp. 106673-106685, doi: 10.1109/ACCESS.2019.2932619.
S. Liu, “Sentiment Analysis of Yelp Reviews: A Comparison of Techniques and Models”, arXiv preprint, arXiv:2004.13851, 2020.
M. R. Huq, A. Ali, and A. Rahman, “Sentiment Analysis on Twitter Data Using KNN and SVM,” International Journal of Advanced Computer Science and Applications, vol. 8, pp. 19-25, 2017, doi: 10.14569/IJACSA.2017.080603.
A. Amolik, N. Jivane, M. Bhandari, and M. Venkatesan “Twitter Sentiment Analysis of Movie Reviews Using Machine Learning Techniques,” International Journal of Engineering and Technology, vol. 7, no. 6, pp. 1-7, 2016.
S. Liao J. Wang R. Yu, K. Sato, and Z., Cheng, “CNN for Situations Understanding Based on Sentiment Analysis of Twitter Data,” Procedia Computer Science, vol. 111, 2017, pp. 376–381, 2017, doi: 10.1016/j.procs.2017.06.037
Li C, Guo X, Mei Q (2017b) Deep Memory Networks for Attitude Identification. In: Proceedings of the tenth ACM International Conference on Web Search and Data Mining, WSDM, Cambridge, United Kingdom, pp 671–680, 2017.
B. Li, Z. Cheng, Z. Xu, W. Ye, T. Lukasiewicz and S. Zhang, “Long Text Analysis Using Sliced Recurrent Neural Networks with Breaking Point Information Enrichment,” ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, United Kingdom, pp. 7550-7554, 2019,doi: 10.1109/ICASSP.2019.8683812.
W. Zhao et al., "Weakly-Supervised Deep Embedding for Product Review Sentiment Analysis," IEEE Transactions on Knowledge and Data Engineering, vol. 30, no. 1, 1 Jan. pp. 185-197, 2018, doi: 10.1109/TKDE.2017.2756658.
M. Al-Smadi, O. Qawasmeh, M. Al-Ayyoub, Y. Jararweh, and B. Gupta, “Deep Recurrent Neural Network vs. Support Vector Machine for Aspect-Based Sentiment Analysis of Arabic Hotels’ Reviews,” Journal of Computational Science, 2017, doi: 10.1016/j.jocs.2017.11.006.
D. Tang, F. Wei, B. Qin, N. Yang, T. Liu, and M. Zhou, “Sentiment Embeddings with Applications to Sentiment Analysis,” In IEEE Transactions on Knowledge and Data Engineering: vol. 28, pp. 496–509, 2016,doi: 10.1109/TKDE.2015.2489653.
P. Chen, Z. Sun, L. Bing, and W. Yang, “Recurrent Attention Network on Memory for Aspect Sentiment Analysis,” Empirical Methods in Natural Language Processing, pp. 452–461, 2017.
F. Tian et al., “Recognizing and Regulating Elearners’ Emotions Based on interactive Chinese Texts in E-Learning Systems,” Knowledge Based System, vol. 55, 148–164, 2014, doi: 10.1016/j.knosys.2013.10.019
H. Ghulam, F. Zeng, W. Li, and Y. Xiao, "Deep learning-based Sentiment Analysis for Roman Urdu Text," Procedia Computer Science, vol. 147, pp.131-135, 2019, doi: 10.1016/j.procs.2019.01.202
J. Singh, R. Singh, and P. Singh, "Morphological evaluation and sentiment analysis of Punjabi text using deep learning classification," Journal King Saud University-Computer and Information Science, 2018, doi: 10.1016/j.jksuci.2018.04.003.
Yelp Polarity Dataset, “TensorFlow Datasets Catalog homepage,” 2015. [online]. Available: https://www.tensorflow.org/datasets/catalog/yelp_polarity_reviews
A. L. Maas, R.E. Daly, P.T. Pham, D. Huang, A.Y. Ng and C. Potts, "Learning Word Vectors for Sentiment Analysis", Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 142-150, 2011.
R. Sjögren, K. Stridh, T. Skotare, J. and J. Trygg, "Multivariate Patent Analysis-Using Chemometrics to Analyze Collections of Chemical and Pharmaceutical Patents," Journal of Chemometrics, vol. 34, pp. e3041, 2020, doi: 10.1002/cem.3041
A. Onan "Mining opinions from instructor evaluation reviews: A Deep Learning Approach, " Computer Application in Engineering Education, vol. 28, pp. 117–138, 2020, doi: 10.1002/cae.22179.
T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space”, arXiv preprint, arXiv:1301.3781, 2013.
T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean, "Distributed Representations of Words and Phrases and Their Compositionality," Neural Information Processing Systems Conference, Lake Tahoe, pp. 3111–3119, 2013.
R. Ni and H. Cao, "Sentiment Analysis based on GloVe and LSTM-GRU," 39th Chinese Control Conference (CCC), Shenyang, China, pp. 7492-7497, 2020, doi: 10.23919/CCC50068.2020.9188578.
M. M. Saritas, A. Yasar, "Performance Analysis of ANN and Naive Bayes Classification Algorithm for Data Classification," International Journal of Intelligent Systems and Applications in Engineering, vol. 7, pp. 88-91,2019, doi: 10.18201//ijisae.2019252786.
S. Qing, H. Wenjie and X. Wenfang, "Robust Support Vector Machine with Bullet Hole Image Classification," IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 32, no. 4, pp. 440-448, 2002, doi: 10.1109/TSMCC.2002.807277.
I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA, USA: MIT Press, 2016.
S. Karita et al., "A Comparative Study on Transformer vs RNN in Speech Applications," IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), SG, Singapore, , pp. 449-456, 2019, doi: 10.1109/ASRU46091.2019.9003750.
L. M. Rojas-Barahona, "Deep Learning for Sentiment Analysis," Language Linguistic Compass, vol. 10, no. 12, 2016, doi: 10.1111/lnc3.12228
Y. LeCun, Y. Bengio, ang G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, 2015, doi: 10.1038/nature14539.
Ş. Kayıkçı,“A convolutional neural network model implementation for speech recognition,” Düzce Üniversitesi Bilim ve Teknoloji Dergisi, vol. 7, no. 3, pp. 1892-1898, 2019, doi: 10.29130/dubited.567828.
M. S. Başarslan and F. Kayaalp, "Performance Analysis Of Fuzzy Rough Set-Based And Correlation-Based Attribute Selection Methods On Detection Of Chronic Kidney Disease With Various Classifiers," 2019 Scientific Meeting on Electrical-Electronics & Biomedical Engineering and Computer Science (EBBT), Istanbul, Turkey, 2019, pp. 1-5. doi: 10.1109/EBBT.2019.8741688.
K. Polat, and S. Güneş, “Breast cancer diagnosis using least square support vector machine,” Digital signal processing, vol. 17, no. 4, pp. 694-701, 2007, doi: 10.1016/j.dsp.2006.10.008.

There are 36 citations in total.

Details

Primary Language	English
Subjects	Artificial Intelligence
Journal Section	Articles
Authors	Muhammet Sinan Başarslan 0000-0002-7996-9169 Fatih Kayaalp 0000-0002-8752-3335
Publication Date	April 30, 2021
Submission Date	November 28, 2020
Acceptance Date	February 4, 2021
Published in Issue	Year 2021Volume: 4 Issue: 1

Cite

IEEE	M. S. Başarslan and F. Kayaalp, “Sentiment Analysis on Social Media Reviews Datasets with Deep Learning Approach”, SAUCIS, vol. 4, no. 1, pp. 35–49, 2021, doi: 10.35377/saucis.04.01.833026.