Research Article
Year 2025,
Volume: 8 Issue: 1, 76 - 88, 28.03.2025
Abstract
Recently, social media has transformed into an essential platform for information dissemination, allowing individuals to articulate their opinions and apprehensions on a wide array of subjects. Stance detection, which refers to the automated examination of text to ascertain the author’s perspective regarding a specific proposition or subject, has emerged as a significant area of research. Within the scope of this study, a Turkish-labeled dataset was created to determine the stances of social media users regarding the Stray Animals Law and various pre-trained BERT models were fine-tuned on this dataset, four of which were Turkish (BERTurk 32k and 128k, ConvBERTurk and ConvBERTurk mC4), one multilingual (mBERT) and one base (BERT-Base). The BERTurk 128k model outperformed other BERT models by achieving a remarkable accuracy rate of 87.10%, along with 87.11% precision, 87.10% recall, and 87.10% F1 score. In conclusion, this study has accomplished a contribution in the limited field of Turkish stance detection research by comparing various BERT models in the context of Turkish texts that has not been previously undertaken to our knowledge. The promising results that were obtained from this and similar studies could contribute to the automatic extraction of public opinions, thereby assisting policymakers in formulating efficient policies.
References
- D. Küçük and F. Can, “Stance detection: A survey,” ACM Computing Surveys (CSUR), vol. 53, no.1, pp. 1-37, 2020.
- S. Mohammad et al., “Semeval-2016 task 6: Detecting stance in tweets,” Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016). 2016.
- R. Cao et al., “Stance detection for online public opinion awareness: An overview,” International Journal of Intelligent Systems, vol. 37 pp. 11944-11965, 2022.
- D. Küçük and F. Can, “Stance detection on tweets: An svm-based approach,” arXiv preprint arXiv:1803.08910, 2018.
- S. Ghosh et al., “Stance detection in web and social media: a comparative study,” Experimental IR Meets Multilinguality, Multimodality, and Interaction: 10th International Conference of the CLEF Association, CLEF 2019, Lugano, Switzerland, September 9–12, 2019, Proceedings 10. Springer International Publishing, 2019. L-A. Cotfas et al., “The longest month: analyzing COVID-19 vaccination opinions dynamics from tweets in the month following the first vaccine announcement,” Ieee Access, vol. 9, pp. 33203-33223, 2021. L. Grimminger and R. Klinger, “Hate towards the political opponent: A Twitter corpus study of the 2020 US elections on the basis of offensive speech and stance detection,” arXiv preprint arXiv:2103.01664, 2021.
- K.K. Polat, N. G. Bayazıt, and O. T. Yıldız, “Türkçe duruş tespit analizi,” Avrupa Bilim ve Teknoloji Dergisi vol. 23, pp.99-107, 2021
- D. Küçük, and N. Arıcı, “Sentiment analysis and stance detection in Turkish tweets about COVID-19 vaccination,” Handbook of research on opinion mining and text analytics on literary works and social media. IGI Global, 371-387, 2022.
- D. Küçük, and N. Arıcı, “Deep learning-based sentiment and stance analysis of Tweets about Vaccination,” International Journal on Semantic Web and Information Systems (IJSWIS), vol. 19, no.1, pp.1-18, 2023.
- M. S. Zengin, B. U. Yenisey, and M. Kutlu, “Exploring the impact of training datasets on Turkish stance detection,” Turkish Journal of Electrical Engineering and Computer Sciences, vol. 31, no.7, pp.1206-1222, 2023.
- S. Arslan and E. Fırat, “Stance Detection on Short Turkish Text: A Case Study of Russia-Ukraine War,” Afyon Kocatepe Üniversitesi Fen Ve Mühendislik Bilimleri Dergisi, vol. 24, no. 3, pp. 602-619, 2024.
- Ekşi Sözlük, “29 temmuz 2024 sokak hayvanları yasası değişikliği,” Available at https://eksisozluk.com/entry/166760652 (Accessed Date: 08.08.2024)
- Ekşi Sözlük, “29 temmuz 2024 sokak hayvanları yasası değişikliği,” Available at https://eksisozluk.com/entry/166761830 (Accessed Date: 08.08.2024)
- J. Devlin, M.-W. Chang, K. Lee and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
- A. Vaswani et al., “Attention is all you need.” Advances in neural information processing systems, 30, 2017.
- P. Savci and B. Das, “Comparison of pre-trained language models in terms of carbon emissions, time and accuracy in multi-label text classification using AutoML,” Heliyon, vol. 9, issue. 5, 2023.
- Hugging Face, “Models,” Available at https://huggingface.co/models?other=bert (Accessed Date: 12.08.2024)
- Z-H. Jiang et al., “Convbert: Improving bert with span-based dynamic convolution,” Advances in Neural Information Processing Systems, vol. 33, pp. 12837-12848, 2020.
- Hugging Face, “dbmdz Turkish ConvBERT model,” Available at https://huggingface.co/dbmdz/convbert-base-turkish-cased (Accessed Date: 12.08.2024)
- X. Chen, P. Cong and S. Lv, “A Long-Text Classification Method of Chinese News Based on BERT and CNN,” Ieee Access, vol. 10, pp. 34046-34057, 2022.
- Medium, “Handle Long Text Corpus for Bert Model,” Available at https://medium.com/@priyatoshanand/handle-long-text-corpus-for-bert-model-3c85248214aa (Accessed Date: 13.08.2024)
- Medium, “Fine-tuning BERT model for arbitrarily long texts, Part 1,” Available at https://medium.com/mim-solutions-blog/fine-tuning-bert-model-for-arbitrarily-long-texts-part-1-299f1533b976 (Accessed Date: 13.08.2024)
- Ekşi Sözlük, “29 temmuz 2024 sokak hayvanları yasası değişikliği”, Available at https://eksisozluk.com/entry/166761830 (Accessed Date: 29.09.2024)
- Y. B. Kaya and A. C. Tantuğ, “Effect of Tokenization Granularity for Turkish Large Language Models,” Journal of Intelligent Systems with Applications, vol. 21, 2024.
- M. T. Ribeiro, S. Singh, and C. Guestrin, “ Why should I trust you?” Explaining the predictions of any classifier, In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016.
- Ekşi Sözlük, “sahipsiz hayvanlara yönelik kanun teklifi,” Available at https://eksisozluk.com/entry/166275172 (Accessed Date: 23.09.2024)
- Ekşi Sözlük, “sokak hayvanları uyutulacak,” Available at https://eksisozluk.com/entry/164628545 (Accessed Date: 23.09.2024)
- Ekşi Sözlük, “sokak hayvanları uyutulacak,” Available at https://eksisozluk.com/entry/164641295 (Accessed Date: 11.01.2025)
- Ekşi Sözlük, “sahipsiz hayvanlara yönelik kanun teklifi,” Available at https://eksisozluk.com/entry/166318051 (Accessed Date: 11.01.2025)
- Ekşi Sözlük, “14 günde sahiplenilmeyen köpeklerin uyutulması,” Available at https://eksisozluk.com/entry/159777438 (Accessed Date: 11.01.2025)
- Ekşi Sözlük, “sokak hayvanları uyutulacak”, Available at https://eksisozluk.com/entry/164651341 (Accessed Date: 11.01.2025)
Year 2025,
Volume: 8 Issue: 1, 76 - 88, 28.03.2025
Abstract
References
- D. Küçük and F. Can, “Stance detection: A survey,” ACM Computing Surveys (CSUR), vol. 53, no.1, pp. 1-37, 2020.
- S. Mohammad et al., “Semeval-2016 task 6: Detecting stance in tweets,” Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016). 2016.
- R. Cao et al., “Stance detection for online public opinion awareness: An overview,” International Journal of Intelligent Systems, vol. 37 pp. 11944-11965, 2022.
- D. Küçük and F. Can, “Stance detection on tweets: An svm-based approach,” arXiv preprint arXiv:1803.08910, 2018.
- S. Ghosh et al., “Stance detection in web and social media: a comparative study,” Experimental IR Meets Multilinguality, Multimodality, and Interaction: 10th International Conference of the CLEF Association, CLEF 2019, Lugano, Switzerland, September 9–12, 2019, Proceedings 10. Springer International Publishing, 2019. L-A. Cotfas et al., “The longest month: analyzing COVID-19 vaccination opinions dynamics from tweets in the month following the first vaccine announcement,” Ieee Access, vol. 9, pp. 33203-33223, 2021. L. Grimminger and R. Klinger, “Hate towards the political opponent: A Twitter corpus study of the 2020 US elections on the basis of offensive speech and stance detection,” arXiv preprint arXiv:2103.01664, 2021.
- K.K. Polat, N. G. Bayazıt, and O. T. Yıldız, “Türkçe duruş tespit analizi,” Avrupa Bilim ve Teknoloji Dergisi vol. 23, pp.99-107, 2021
- D. Küçük, and N. Arıcı, “Sentiment analysis and stance detection in Turkish tweets about COVID-19 vaccination,” Handbook of research on opinion mining and text analytics on literary works and social media. IGI Global, 371-387, 2022.
- D. Küçük, and N. Arıcı, “Deep learning-based sentiment and stance analysis of Tweets about Vaccination,” International Journal on Semantic Web and Information Systems (IJSWIS), vol. 19, no.1, pp.1-18, 2023.
- M. S. Zengin, B. U. Yenisey, and M. Kutlu, “Exploring the impact of training datasets on Turkish stance detection,” Turkish Journal of Electrical Engineering and Computer Sciences, vol. 31, no.7, pp.1206-1222, 2023.
- S. Arslan and E. Fırat, “Stance Detection on Short Turkish Text: A Case Study of Russia-Ukraine War,” Afyon Kocatepe Üniversitesi Fen Ve Mühendislik Bilimleri Dergisi, vol. 24, no. 3, pp. 602-619, 2024.
- Ekşi Sözlük, “29 temmuz 2024 sokak hayvanları yasası değişikliği,” Available at https://eksisozluk.com/entry/166760652 (Accessed Date: 08.08.2024)
- Ekşi Sözlük, “29 temmuz 2024 sokak hayvanları yasası değişikliği,” Available at https://eksisozluk.com/entry/166761830 (Accessed Date: 08.08.2024)
- J. Devlin, M.-W. Chang, K. Lee and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
- A. Vaswani et al., “Attention is all you need.” Advances in neural information processing systems, 30, 2017.
- P. Savci and B. Das, “Comparison of pre-trained language models in terms of carbon emissions, time and accuracy in multi-label text classification using AutoML,” Heliyon, vol. 9, issue. 5, 2023.
- Hugging Face, “Models,” Available at https://huggingface.co/models?other=bert (Accessed Date: 12.08.2024)
- Z-H. Jiang et al., “Convbert: Improving bert with span-based dynamic convolution,” Advances in Neural Information Processing Systems, vol. 33, pp. 12837-12848, 2020.
- Hugging Face, “dbmdz Turkish ConvBERT model,” Available at https://huggingface.co/dbmdz/convbert-base-turkish-cased (Accessed Date: 12.08.2024)
- X. Chen, P. Cong and S. Lv, “A Long-Text Classification Method of Chinese News Based on BERT and CNN,” Ieee Access, vol. 10, pp. 34046-34057, 2022.
- Medium, “Handle Long Text Corpus for Bert Model,” Available at https://medium.com/@priyatoshanand/handle-long-text-corpus-for-bert-model-3c85248214aa (Accessed Date: 13.08.2024)
- Medium, “Fine-tuning BERT model for arbitrarily long texts, Part 1,” Available at https://medium.com/mim-solutions-blog/fine-tuning-bert-model-for-arbitrarily-long-texts-part-1-299f1533b976 (Accessed Date: 13.08.2024)
- Ekşi Sözlük, “29 temmuz 2024 sokak hayvanları yasası değişikliği”, Available at https://eksisozluk.com/entry/166761830 (Accessed Date: 29.09.2024)
- Y. B. Kaya and A. C. Tantuğ, “Effect of Tokenization Granularity for Turkish Large Language Models,” Journal of Intelligent Systems with Applications, vol. 21, 2024.
- M. T. Ribeiro, S. Singh, and C. Guestrin, “ Why should I trust you?” Explaining the predictions of any classifier, In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016.
- Ekşi Sözlük, “sahipsiz hayvanlara yönelik kanun teklifi,” Available at https://eksisozluk.com/entry/166275172 (Accessed Date: 23.09.2024)
- Ekşi Sözlük, “sokak hayvanları uyutulacak,” Available at https://eksisozluk.com/entry/164628545 (Accessed Date: 23.09.2024)
- Ekşi Sözlük, “sokak hayvanları uyutulacak,” Available at https://eksisozluk.com/entry/164641295 (Accessed Date: 11.01.2025)
- Ekşi Sözlük, “sahipsiz hayvanlara yönelik kanun teklifi,” Available at https://eksisozluk.com/entry/166318051 (Accessed Date: 11.01.2025)
- Ekşi Sözlük, “14 günde sahiplenilmeyen köpeklerin uyutulması,” Available at https://eksisozluk.com/entry/159777438 (Accessed Date: 11.01.2025)
- Ekşi Sözlük, “sokak hayvanları uyutulacak”, Available at https://eksisozluk.com/entry/164651341 (Accessed Date: 11.01.2025)
There are 30 citations in total.
Details
| Primary Language | English |
|---|---|
| Subjects | Software Engineering (Other) |
| Journal Section | Research Article |
| Authors | |
| Early Pub Date | March 27, 2025 |
| Publication Date | March 28, 2025 |
| Submission Date | October 9, 2024 |
| Acceptance Date | March 5, 2025 |
| Published in Issue | Year 2025Volume: 8 Issue: 1 |
Cite
INDEXING & ABSTRACTING & ARCHIVING
The papers in this journal are licensed under a Creative Commons Attribution-NonCommercial 4.0 International License