Turkish Stance Detection on Social Media Using BERT Models: A Case Study of Stray Animals Law

Selma Alav; Kristin Surpuhi Benli

doi:10.35377/saucis...1564138

EN

Turkish Stance Detection on Social Media Using BERT Models: A Case Study of Stray Animals Law

Abstract

Recently, social media has transformed into an essential platform for information dissemination, allowing individuals to articulate their opinions and apprehensions on a wide array of subjects. Stance detection, which refers to the automated examination of text to ascertain the author’s perspective regarding a specific proposition or subject, has emerged as a significant area of research. Within the scope of this study, a Turkish-labeled dataset was created to determine the stances of social media users regarding the Stray Animals Law and various pre-trained BERT models were fine-tuned on this dataset, four of which were Turkish (BERTurk 32k and 128k, ConvBERTurk and ConvBERTurk mC4), one multilingual (mBERT) and one base (BERT-Base). The BERTurk 128k model outperformed other BERT models by achieving a remarkable accuracy rate of 87.10%, along with 87.11% precision, 87.10% recall, and 87.10% F1 score. In conclusion, this study has accomplished a contribution in the limited field of Turkish stance detection research by comparing various BERT models in the context of Turkish texts that has not been previously undertaken to our knowledge. The promising results that were obtained from this and similar studies could contribute to the automatic extraction of public opinions, thereby assisting policymakers in formulating efficient policies.

Keywords

References

D. Küçük and F. Can, “Stance detection: A survey,” ACM Computing Surveys (CSUR), vol. 53, no.1, pp. 1-37, 2020.
S. Mohammad et al., “Semeval-2016 task 6: Detecting stance in tweets,” Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016). 2016.
R. Cao et al., “Stance detection for online public opinion awareness: An overview,” International Journal of Intelligent Systems, vol. 37 pp. 11944-11965, 2022.
D. Küçük and F. Can, “Stance detection on tweets: An svm-based approach,” arXiv preprint arXiv:1803.08910, 2018.
S. Ghosh et al., “Stance detection in web and social media: a comparative study,” Experimental IR Meets Multilinguality, Multimodality, and Interaction: 10th International Conference of the CLEF Association, CLEF 2019, Lugano, Switzerland, September 9–12, 2019, Proceedings 10. Springer International Publishing, 2019. L-A. Cotfas et al., “The longest month: analyzing COVID-19 vaccination opinions dynamics from tweets in the month following the first vaccine announcement,” Ieee Access, vol. 9, pp. 33203-33223, 2021. L. Grimminger and R. Klinger, “Hate towards the political opponent: A Twitter corpus study of the 2020 US elections on the basis of offensive speech and stance detection,” arXiv preprint arXiv:2103.01664, 2021.
K.K. Polat, N. G. Bayazıt, and O. T. Yıldız, “Türkçe duruş tespit analizi,” Avrupa Bilim ve Teknoloji Dergisi vol. 23, pp.99-107, 2021
D. Küçük, and N. Arıcı, “Sentiment analysis and stance detection in Turkish tweets about COVID-19 vaccination,” Handbook of research on opinion mining and text analytics on literary works and social media. IGI Global, 371-387, 2022.
D. Küçük, and N. Arıcı, “Deep learning-based sentiment and stance analysis of Tweets about Vaccination,” International Journal on Semantic Web and Information Systems (IJSWIS), vol. 19, no.1, pp.1-18, 2023.

M. S. Zengin, B. U. Yenisey, and M. Kutlu, “Exploring the impact of training datasets on Turkish stance detection,” Turkish Journal of Electrical Engineering and Computer Sciences, vol. 31, no.7, pp.1206-1222, 2023.
S. Arslan and E. Fırat, “Stance Detection on Short Turkish Text: A Case Study of Russia-Ukraine War,” Afyon Kocatepe Üniversitesi Fen Ve Mühendislik Bilimleri Dergisi, vol. 24, no. 3, pp. 602-619, 2024.
Ekşi Sözlük, “29 temmuz 2024 sokak hayvanları yasası değişikliği,” Available at https://eksisozluk.com/entry/166760652 (Accessed Date: 08.08.2024)
Ekşi Sözlük, “29 temmuz 2024 sokak hayvanları yasası değişikliği,” Available at https://eksisozluk.com/entry/166761830 (Accessed Date: 08.08.2024)
J. Devlin, M.-W. Chang, K. Lee and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
A. Vaswani et al., “Attention is all you need.” Advances in neural information processing systems, 30, 2017.
P. Savci and B. Das, “Comparison of pre-trained language models in terms of carbon emissions, time and accuracy in multi-label text classification using AutoML,” Heliyon, vol. 9, issue. 5, 2023.
Hugging Face, “Models,” Available at https://huggingface.co/models?other=bert (Accessed Date: 12.08.2024)
Z-H. Jiang et al., “Convbert: Improving bert with span-based dynamic convolution,” Advances in Neural Information Processing Systems, vol. 33, pp. 12837-12848, 2020.
Hugging Face, “dbmdz Turkish ConvBERT model,” Available at https://huggingface.co/dbmdz/convbert-base-turkish-cased (Accessed Date: 12.08.2024)
X. Chen, P. Cong and S. Lv, “A Long-Text Classification Method of Chinese News Based on BERT and CNN,” Ieee Access, vol. 10, pp. 34046-34057, 2022.
Medium, “Handle Long Text Corpus for Bert Model,” Available at https://medium.com/@priyatoshanand/handle-long-text-corpus-for-bert-model-3c85248214aa (Accessed Date: 13.08.2024)
Medium, “Fine-tuning BERT model for arbitrarily long texts, Part 1,” Available at https://medium.com/mim-solutions-blog/fine-tuning-bert-model-for-arbitrarily-long-texts-part-1-299f1533b976 (Accessed Date: 13.08.2024)
Ekşi Sözlük, “29 temmuz 2024 sokak hayvanları yasası değişikliği”, Available at https://eksisozluk.com/entry/166761830 (Accessed Date: 29.09.2024)
Y. B. Kaya and A. C. Tantuğ, “Eﬀect of Tokenization Granularity for Turkish Large Language Models,” Journal of Intelligent Systems with Applications, vol. 21, 2024.
M. T. Ribeiro, S. Singh, and C. Guestrin, “ Why should I trust you?” Explaining the predictions of any classifier, In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016.
Ekşi Sözlük, “sahipsiz hayvanlara yönelik kanun teklifi,” Available at https://eksisozluk.com/entry/166275172 (Accessed Date: 23.09.2024)
Ekşi Sözlük, “sokak hayvanları uyutulacak,” Available at https://eksisozluk.com/entry/164628545 (Accessed Date: 23.09.2024)
Ekşi Sözlük, “sokak hayvanları uyutulacak,” Available at https://eksisozluk.com/entry/164641295 (Accessed Date: 11.01.2025)
Ekşi Sözlük, “sahipsiz hayvanlara yönelik kanun teklifi,” Available at https://eksisozluk.com/entry/166318051 (Accessed Date: 11.01.2025)
Ekşi Sözlük, “14 günde sahiplenilmeyen köpeklerin uyutulması,” Available at https://eksisozluk.com/entry/159777438 (Accessed Date: 11.01.2025)
Ekşi Sözlük, “sokak hayvanları uyutulacak”, Available at https://eksisozluk.com/entry/164651341 (Accessed Date: 11.01.2025)

Details

Primary Language

English

Subjects

Software Engineering (Other)

Journal Section

Research Article

Authors

Selma Alav
0009-0009-1521-032X
Türkiye

Kristin Surpuhi Benli ^*
0000-0001-6282-6703
Türkiye

Early Pub Date

March 27, 2025

Publication Date

March 28, 2025

Submission Date

October 9, 2024

Acceptance Date

March 5, 2025

Published in Issue

Year 2025 Volume: 8 Number: 1

DOI

https://doi.org/10.35377/saucis...1564138

IZ

https://izlik.org/JA45NJ76PC

Cite

RIS / Bibtex

APA

Alav, S., & Benli, K. S. (2025). Turkish Stance Detection on Social Media Using BERT Models: A Case Study of Stray Animals Law. Sakarya University Journal of Computer and Information Sciences, 8(1), 76-88. https://doi.org/10.35377/saucis...1564138

AMA

1.Alav S, Benli KS. Turkish Stance Detection on Social Media Using BERT Models: A Case Study of Stray Animals Law. SAUCIS. 2025;8(1):76-88. doi:10.35377/saucis.1564138

Chicago

Alav, Selma, and Kristin Surpuhi Benli. 2025. “Turkish Stance Detection on Social Media Using BERT Models: A Case Study of Stray Animals Law”. Sakarya University Journal of Computer and Information Sciences 8 (1): 76-88. https://doi.org/10.35377/saucis. 1564138.

EndNote

Alav S, Benli KS (March 1, 2025) Turkish Stance Detection on Social Media Using BERT Models: A Case Study of Stray Animals Law. Sakarya University Journal of Computer and Information Sciences 8 1 76–88.

IEEE

[1]S. Alav and K. S. Benli, “Turkish Stance Detection on Social Media Using BERT Models: A Case Study of Stray Animals Law”, SAUCIS, vol. 8, no. 1, pp. 76–88, Mar. 2025, doi: 10.35377/saucis...1564138.

ISNAD

Alav, Selma - Benli, Kristin Surpuhi. “Turkish Stance Detection on Social Media Using BERT Models: A Case Study of Stray Animals Law”. Sakarya University Journal of Computer and Information Sciences 8/1 (March 1, 2025): 76-88. https://doi.org/10.35377/saucis. 1564138.

JAMA

1.Alav S, Benli KS. Turkish Stance Detection on Social Media Using BERT Models: A Case Study of Stray Animals Law. SAUCIS. 2025;8:76–88.

MLA

Alav, Selma, and Kristin Surpuhi Benli. “Turkish Stance Detection on Social Media Using BERT Models: A Case Study of Stray Animals Law”. Sakarya University Journal of Computer and Information Sciences, vol. 8, no. 1, Mar. 2025, pp. 76-88, doi:10.35377/saucis. 1564138.

Vancouver

1.Selma Alav, Kristin Surpuhi Benli. Turkish Stance Detection on Social Media Using BERT Models: A Case Study of Stray Animals Law. SAUCIS. 2025 Mar. 1;8(1):76-88. doi:10.35377/saucis. 1564138