Araştırma Makalesi
PDF Zotero Mendeley EndNote BibTex Kaynak Göster

Sentiment Analysis on Social Media Reviews Datasets with Deep Learning Approach

Yıl 2021, Cilt 4, Sayı 1, 35 - 49, 30.04.2021
https://doi.org/10.35377/saucis.04.01.833026

Öz

Thanks to social media, people are now able to leave guiding comments quickly about their favorite restaurants, movies, etc. This has paved the way for the field of sentiment analysis, which brings together various disciplines. In this study, Yelp restaurant reviews and IMDB movie reviews dataset were used together with the data collected from Twitter. Word2Vec (W2V), Global Vector (GloVe) and Bidirectional Encoder Representation (BERT) word embedding methods, Term Frequency-Reverse Document Frequency (TF-IDF), and the Bag-of-Words (BOW) were used on these datasets. Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Recurrent Neural Network (RNN), Support Vector Machine (SVM), and Naive Bayes (NB) were used in the sentiment analysis models. Accuracy, F-measure (F), Sensitivity (Sens), Precision (Pre), and Receiver Operating Characteristics (ROC) were used in the evaluation of the model performance. The Accuracy rates of the models created by the Machine Learning (ML) and Deep Learning (DL) methods using the IMDB dataset were in the range of 81%-90% and 84%-94%, respectively. These rates were in the range of 80%-86% and 81%-89% for the Yelp dataset, and in the range of 75%-79% and 85%-98% for the Twitter dataset. The models that incorporated the BERT word embedding method have the best performance, compared to the other models with ML and DL. Therefore, BERT method is recommended for this type of analysis in future studies.

Kaynakça

  • E. Park, J. Kang, D. Choi, and J. Han, “Understanding Customers' Hotel Revisiting Behaviour: a sentiment analysis of Online Feedback Reviews,” Current Issues in Tourism, vol. 23, pp. 605-611, 2020, doi: 10.1080/13683500.2018.1549025.
  • B. Pang and L. Lee, "Opinion mining and sentiment analysis", Foundations Trends Information Retrival, vol. 2, no. 2, 2008, pp. 1-135.
  • O. Kaynar, H. Arslan, Y. Görmez and F. Demirkoparan, "Makine Öğrenmesi Yöntemleri ile Duygu Analizi," International Artificial Intelligence and Data Processing Symposium (IDAP), pp. 1-5, Malatya, 2017.
  • A. Al Hamoud, A. Alwehaibi, K. Roy, and M. Bikdash, “Classifying Political Tweets Using Naïve Bayes and Support Vector Machines,” In International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, pp. 736-744, 2018, doi: 10.1007/978-3-319-92058-0_71.
  • S. Symeonidis, D. Effrosynidis, and A, Arampatzis, “A Comparative Evaluation of Pre‐Processing Techniques and Their Interactions for Twitter Sentiment Analysis,” Expert System Applications, vol. 110, pp. 298-310, 2018, doi: 10.1016/j.eswa.2018.06.022.
  • M. A. Paredes-Valverde, R. Colomo-Palacios, M. P. Salas-Zárate, and R. Valencia-García, “Sentiment Analysis in Spanish for Improvement of Products and Services: A Deep Learning Approach,” Scientific Programming, vol. 2017, 2017, doi: 10.1155/2017/1329281.
  • J. Zheng and L. Zheng, "A Hybrid Bidirectional Recurrent Convolutional Neural Network Attention-Based Model for Text Classification," IEEE Access, vol. 7, 2019, pp. 106673-106685, doi: 10.1109/ACCESS.2019.2932619.
  • S. Liu, “Sentiment Analysis of Yelp Reviews: A Comparison of Techniques and Models”, arXiv preprint, arXiv:2004.13851, 2020.
  • M. R. Huq, A. Ali, and A. Rahman, “Sentiment Analysis on Twitter Data Using KNN and SVM,” International Journal of Advanced Computer Science and Applications, vol. 8, pp. 19-25, 2017, doi: 10.14569/IJACSA.2017.080603.
  • A. Amolik, N. Jivane, M. Bhandari, and M. Venkatesan “Twitter Sentiment Analysis of Movie Reviews Using Machine Learning Techniques,” International Journal of Engineering and Technology, vol. 7, no. 6, pp. 1-7, 2016.
  • S. Liao J. Wang R. Yu, K. Sato, and Z., Cheng, “CNN for Situations Understanding Based on Sentiment Analysis of Twitter Data,” Procedia Computer Science, vol. 111, 2017, pp. 376–381, 2017, doi: 10.1016/j.procs.2017.06.037
  • Li C, Guo X, Mei Q (2017b) Deep Memory Networks for Attitude Identification. In: Proceedings of the tenth ACM International Conference on Web Search and Data Mining, WSDM, Cambridge, United Kingdom, pp 671–680, 2017.
  • B. Li, Z. Cheng, Z. Xu, W. Ye, T. Lukasiewicz and S. Zhang, “Long Text Analysis Using Sliced Recurrent Neural Networks with Breaking Point Information Enrichment,” ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, United Kingdom, pp. 7550-7554, 2019,doi: 10.1109/ICASSP.2019.8683812.
  • W. Zhao et al., "Weakly-Supervised Deep Embedding for Product Review Sentiment Analysis," IEEE Transactions on Knowledge and Data Engineering, vol. 30, no. 1, 1 Jan. pp. 185-197, 2018, doi: 10.1109/TKDE.2017.2756658.
  • M. Al-Smadi, O. Qawasmeh, M. Al-Ayyoub, Y. Jararweh, and B. Gupta, “Deep Recurrent Neural Network vs. Support Vector Machine for Aspect-Based Sentiment Analysis of Arabic Hotels’ Reviews,” Journal of Computational Science, 2017, doi: 10.1016/j.jocs.2017.11.006.
  • D. Tang, F. Wei, B. Qin, N. Yang, T. Liu, and M. Zhou, “Sentiment Embeddings with Applications to Sentiment Analysis,” In IEEE Transactions on Knowledge and Data Engineering: vol. 28, pp. 496–509, 2016,doi: 10.1109/TKDE.2015.2489653.
  • P. Chen, Z. Sun, L. Bing, and W. Yang, “Recurrent Attention Network on Memory for Aspect Sentiment Analysis,” Empirical Methods in Natural Language Processing, pp. 452–461, 2017.
  • F. Tian et al., “Recognizing and Regulating Elearners’ Emotions Based on interactive Chinese Texts in E-Learning Systems,” Knowledge Based System, vol. 55, 148–164, 2014, doi: 10.1016/j.knosys.2013.10.019
  • H. Ghulam, F. Zeng, W. Li, and Y. Xiao, "Deep learning-based Sentiment Analysis for Roman Urdu Text," Procedia Computer Science, vol. 147, pp.131-135, 2019, doi: 10.1016/j.procs.2019.01.202
  • J. Singh, R. Singh, and P. Singh, "Morphological evaluation and sentiment analysis of Punjabi text using deep learning classification," Journal King Saud University-Computer and Information Science, 2018, doi: 10.1016/j.jksuci.2018.04.003.
  • Yelp Polarity Dataset, “TensorFlow Datasets Catalog homepage,” 2015. [online]. Available: https://www.tensorflow.org/datasets/catalog/yelp_polarity_reviews
  • A. L. Maas, R.E. Daly, P.T. Pham, D. Huang, A.Y. Ng and C. Potts, "Learning Word Vectors for Sentiment Analysis", Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 142-150, 2011.
  • R. Sjögren, K. Stridh, T. Skotare, J. and J. Trygg, "Multivariate Patent Analysis-Using Chemometrics to Analyze Collections of Chemical and Pharmaceutical Patents," Journal of Chemometrics, vol. 34, pp. e3041, 2020, doi: 10.1002/cem.3041
  • A. Onan "Mining opinions from instructor evaluation reviews: A Deep Learning Approach, " Computer Application in Engineering Education, vol. 28, pp. 117–138, 2020, doi: 10.1002/cae.22179.
  • T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space”, arXiv preprint, arXiv:1301.3781, 2013.
  • T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean, "Distributed Representations of Words and Phrases and Their Compositionality," Neural Information Processing Systems Conference, Lake Tahoe, pp. 3111–3119, 2013.
  • R. Ni and H. Cao, "Sentiment Analysis based on GloVe and LSTM-GRU," 39th Chinese Control Conference (CCC), Shenyang, China, pp. 7492-7497, 2020, doi: 10.23919/CCC50068.2020.9188578.
  • M. M. Saritas, A. Yasar, "Performance Analysis of ANN and Naive Bayes Classification Algorithm for Data Classification," International Journal of Intelligent Systems and Applications in Engineering, vol. 7, pp. 88-91,2019, doi: 10.18201//ijisae.2019252786.
  • S. Qing, H. Wenjie and X. Wenfang, "Robust Support Vector Machine with Bullet Hole Image Classification," IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 32, no. 4, pp. 440-448, 2002, doi: 10.1109/TSMCC.2002.807277.
  • I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA, USA: MIT Press, 2016.
  • S. Karita et al., "A Comparative Study on Transformer vs RNN in Speech Applications," IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), SG, Singapore, , pp. 449-456, 2019, doi: 10.1109/ASRU46091.2019.9003750.
  • L. M. Rojas-Barahona, "Deep Learning for Sentiment Analysis," Language Linguistic Compass, vol. 10, no. 12, 2016, doi: 10.1111/lnc3.12228
  • Y. LeCun, Y. Bengio, ang G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, 2015, doi: 10.1038/nature14539.
  • Ş. Kayıkçı,“A convolutional neural network model implementation for speech recognition,” Düzce Üniversitesi Bilim ve Teknoloji Dergisi, vol. 7, no. 3, pp. 1892-1898, 2019, doi: 10.29130/dubited.567828.
  • M. S. Başarslan and F. Kayaalp, "Performance Analysis Of Fuzzy Rough Set-Based And Correlation-Based Attribute Selection Methods On Detection Of Chronic Kidney Disease With Various Classifiers," 2019 Scientific Meeting on Electrical-Electronics & Biomedical Engineering and Computer Science (EBBT), Istanbul, Turkey, 2019, pp. 1-5. doi: 10.1109/EBBT.2019.8741688.
  • K. Polat, and S. Güneş, “Breast cancer diagnosis using least square support vector machine,” Digital signal processing, vol. 17, no. 4, pp. 694-701, 2007, doi: 10.1016/j.dsp.2006.10.008.

Yıl 2021, Cilt 4, Sayı 1, 35 - 49, 30.04.2021
https://doi.org/10.35377/saucis.04.01.833026

Öz

Kaynakça

  • E. Park, J. Kang, D. Choi, and J. Han, “Understanding Customers' Hotel Revisiting Behaviour: a sentiment analysis of Online Feedback Reviews,” Current Issues in Tourism, vol. 23, pp. 605-611, 2020, doi: 10.1080/13683500.2018.1549025.
  • B. Pang and L. Lee, "Opinion mining and sentiment analysis", Foundations Trends Information Retrival, vol. 2, no. 2, 2008, pp. 1-135.
  • O. Kaynar, H. Arslan, Y. Görmez and F. Demirkoparan, "Makine Öğrenmesi Yöntemleri ile Duygu Analizi," International Artificial Intelligence and Data Processing Symposium (IDAP), pp. 1-5, Malatya, 2017.
  • A. Al Hamoud, A. Alwehaibi, K. Roy, and M. Bikdash, “Classifying Political Tweets Using Naïve Bayes and Support Vector Machines,” In International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, pp. 736-744, 2018, doi: 10.1007/978-3-319-92058-0_71.
  • S. Symeonidis, D. Effrosynidis, and A, Arampatzis, “A Comparative Evaluation of Pre‐Processing Techniques and Their Interactions for Twitter Sentiment Analysis,” Expert System Applications, vol. 110, pp. 298-310, 2018, doi: 10.1016/j.eswa.2018.06.022.
  • M. A. Paredes-Valverde, R. Colomo-Palacios, M. P. Salas-Zárate, and R. Valencia-García, “Sentiment Analysis in Spanish for Improvement of Products and Services: A Deep Learning Approach,” Scientific Programming, vol. 2017, 2017, doi: 10.1155/2017/1329281.
  • J. Zheng and L. Zheng, "A Hybrid Bidirectional Recurrent Convolutional Neural Network Attention-Based Model for Text Classification," IEEE Access, vol. 7, 2019, pp. 106673-106685, doi: 10.1109/ACCESS.2019.2932619.
  • S. Liu, “Sentiment Analysis of Yelp Reviews: A Comparison of Techniques and Models”, arXiv preprint, arXiv:2004.13851, 2020.
  • M. R. Huq, A. Ali, and A. Rahman, “Sentiment Analysis on Twitter Data Using KNN and SVM,” International Journal of Advanced Computer Science and Applications, vol. 8, pp. 19-25, 2017, doi: 10.14569/IJACSA.2017.080603.
  • A. Amolik, N. Jivane, M. Bhandari, and M. Venkatesan “Twitter Sentiment Analysis of Movie Reviews Using Machine Learning Techniques,” International Journal of Engineering and Technology, vol. 7, no. 6, pp. 1-7, 2016.
  • S. Liao J. Wang R. Yu, K. Sato, and Z., Cheng, “CNN for Situations Understanding Based on Sentiment Analysis of Twitter Data,” Procedia Computer Science, vol. 111, 2017, pp. 376–381, 2017, doi: 10.1016/j.procs.2017.06.037
  • Li C, Guo X, Mei Q (2017b) Deep Memory Networks for Attitude Identification. In: Proceedings of the tenth ACM International Conference on Web Search and Data Mining, WSDM, Cambridge, United Kingdom, pp 671–680, 2017.
  • B. Li, Z. Cheng, Z. Xu, W. Ye, T. Lukasiewicz and S. Zhang, “Long Text Analysis Using Sliced Recurrent Neural Networks with Breaking Point Information Enrichment,” ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, United Kingdom, pp. 7550-7554, 2019,doi: 10.1109/ICASSP.2019.8683812.
  • W. Zhao et al., "Weakly-Supervised Deep Embedding for Product Review Sentiment Analysis," IEEE Transactions on Knowledge and Data Engineering, vol. 30, no. 1, 1 Jan. pp. 185-197, 2018, doi: 10.1109/TKDE.2017.2756658.
  • M. Al-Smadi, O. Qawasmeh, M. Al-Ayyoub, Y. Jararweh, and B. Gupta, “Deep Recurrent Neural Network vs. Support Vector Machine for Aspect-Based Sentiment Analysis of Arabic Hotels’ Reviews,” Journal of Computational Science, 2017, doi: 10.1016/j.jocs.2017.11.006.
  • D. Tang, F. Wei, B. Qin, N. Yang, T. Liu, and M. Zhou, “Sentiment Embeddings with Applications to Sentiment Analysis,” In IEEE Transactions on Knowledge and Data Engineering: vol. 28, pp. 496–509, 2016,doi: 10.1109/TKDE.2015.2489653.
  • P. Chen, Z. Sun, L. Bing, and W. Yang, “Recurrent Attention Network on Memory for Aspect Sentiment Analysis,” Empirical Methods in Natural Language Processing, pp. 452–461, 2017.
  • F. Tian et al., “Recognizing and Regulating Elearners’ Emotions Based on interactive Chinese Texts in E-Learning Systems,” Knowledge Based System, vol. 55, 148–164, 2014, doi: 10.1016/j.knosys.2013.10.019
  • H. Ghulam, F. Zeng, W. Li, and Y. Xiao, "Deep learning-based Sentiment Analysis for Roman Urdu Text," Procedia Computer Science, vol. 147, pp.131-135, 2019, doi: 10.1016/j.procs.2019.01.202
  • J. Singh, R. Singh, and P. Singh, "Morphological evaluation and sentiment analysis of Punjabi text using deep learning classification," Journal King Saud University-Computer and Information Science, 2018, doi: 10.1016/j.jksuci.2018.04.003.
  • Yelp Polarity Dataset, “TensorFlow Datasets Catalog homepage,” 2015. [online]. Available: https://www.tensorflow.org/datasets/catalog/yelp_polarity_reviews
  • A. L. Maas, R.E. Daly, P.T. Pham, D. Huang, A.Y. Ng and C. Potts, "Learning Word Vectors for Sentiment Analysis", Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 142-150, 2011.
  • R. Sjögren, K. Stridh, T. Skotare, J. and J. Trygg, "Multivariate Patent Analysis-Using Chemometrics to Analyze Collections of Chemical and Pharmaceutical Patents," Journal of Chemometrics, vol. 34, pp. e3041, 2020, doi: 10.1002/cem.3041
  • A. Onan "Mining opinions from instructor evaluation reviews: A Deep Learning Approach, " Computer Application in Engineering Education, vol. 28, pp. 117–138, 2020, doi: 10.1002/cae.22179.
  • T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space”, arXiv preprint, arXiv:1301.3781, 2013.
  • T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean, "Distributed Representations of Words and Phrases and Their Compositionality," Neural Information Processing Systems Conference, Lake Tahoe, pp. 3111–3119, 2013.
  • R. Ni and H. Cao, "Sentiment Analysis based on GloVe and LSTM-GRU," 39th Chinese Control Conference (CCC), Shenyang, China, pp. 7492-7497, 2020, doi: 10.23919/CCC50068.2020.9188578.
  • M. M. Saritas, A. Yasar, "Performance Analysis of ANN and Naive Bayes Classification Algorithm for Data Classification," International Journal of Intelligent Systems and Applications in Engineering, vol. 7, pp. 88-91,2019, doi: 10.18201//ijisae.2019252786.
  • S. Qing, H. Wenjie and X. Wenfang, "Robust Support Vector Machine with Bullet Hole Image Classification," IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 32, no. 4, pp. 440-448, 2002, doi: 10.1109/TSMCC.2002.807277.
  • I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA, USA: MIT Press, 2016.
  • S. Karita et al., "A Comparative Study on Transformer vs RNN in Speech Applications," IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), SG, Singapore, , pp. 449-456, 2019, doi: 10.1109/ASRU46091.2019.9003750.
  • L. M. Rojas-Barahona, "Deep Learning for Sentiment Analysis," Language Linguistic Compass, vol. 10, no. 12, 2016, doi: 10.1111/lnc3.12228
  • Y. LeCun, Y. Bengio, ang G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, 2015, doi: 10.1038/nature14539.
  • Ş. Kayıkçı,“A convolutional neural network model implementation for speech recognition,” Düzce Üniversitesi Bilim ve Teknoloji Dergisi, vol. 7, no. 3, pp. 1892-1898, 2019, doi: 10.29130/dubited.567828.
  • M. S. Başarslan and F. Kayaalp, "Performance Analysis Of Fuzzy Rough Set-Based And Correlation-Based Attribute Selection Methods On Detection Of Chronic Kidney Disease With Various Classifiers," 2019 Scientific Meeting on Electrical-Electronics & Biomedical Engineering and Computer Science (EBBT), Istanbul, Turkey, 2019, pp. 1-5. doi: 10.1109/EBBT.2019.8741688.
  • K. Polat, and S. Güneş, “Breast cancer diagnosis using least square support vector machine,” Digital signal processing, vol. 17, no. 4, pp. 694-701, 2007, doi: 10.1016/j.dsp.2006.10.008.

Ayrıntılar

Birincil Dil İngilizce
Konular Bilgisayar Bilimleri, Yapay Zeka
Bölüm Makaleler
Yazarlar

Muhammet Sinan BAŞARSLAN (Sorumlu Yazar)
DOĞUŞ ÜNİVERSİTESİ
0000-0002-7996-9169
Türkiye


Fatih KAYAALP
DUZCE UNIVERSITY
0000-0002-8752-3335
Türkiye

Yayımlanma Tarihi 30 Nisan 2021
Başvuru Tarihi 28 Kasım 2020
Kabul Tarihi 4 Şubat 2021
Yayınlandığı Sayı Yıl 2021, Cilt 4, Sayı 1

Kaynak Göster

IEEE M. S. Başarslan ve F. Kayaalp , "Sentiment Analysis on Social Media Reviews Datasets with Deep Learning Approach", Sakarya University Journal of Computer and Information Sciences, c. 4, sayı. 1, ss. 35-49, Nis. 2021, doi:10.35377/saucis.04.01.833026