Loading [a11y]/accessibility-menu.js
Research Article
BibTex RIS Cite

Turkish Stance Detection on Social Media Using BERT Models: A Case Study of Stray Animals Law

Year 2025, Volume: 8 Issue: 1, 76 - 88, 28.03.2025
https://doi.org/10.35377/saucis...1564138

Abstract

Recently, social media has transformed into an essential platform for information dissemination, allowing individuals to articulate their opinions and apprehensions on a wide array of subjects. Stance detection, which refers to the automated examination of text to ascertain the author’s perspective regarding a specific proposition or subject, has emerged as a significant area of research. Within the scope of this study, a Turkish-labeled dataset was created to determine the stances of social media users regarding the Stray Animals Law and various pre-trained BERT models were fine-tuned on this dataset, four of which were Turkish (BERTurk 32k and 128k, ConvBERTurk and ConvBERTurk mC4), one multilingual (mBERT) and one base (BERT-Base). The BERTurk 128k model outperformed other BERT models by achieving a remarkable accuracy rate of 87.10%, along with 87.11% precision, 87.10% recall, and 87.10% F1 score. In conclusion, this study has accomplished a contribution in the limited field of Turkish stance detection research by comparing various BERT models in the context of Turkish texts that has not been previously undertaken to our knowledge. The promising results that were obtained from this and similar studies could contribute to the automatic extraction of public opinions, thereby assisting policymakers in formulating efficient policies.

References

  • D. Küçük and F. Can, “Stance detection: A survey,” ACM Computing Surveys (CSUR), vol. 53, no.1, pp. 1-37, 2020.
  • S. Mohammad et al., “Semeval-2016 task 6: Detecting stance in tweets,” Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016). 2016.
  • R. Cao et al., “Stance detection for online public opinion awareness: An overview,” International Journal of Intelligent Systems, vol. 37 pp. 11944-11965, 2022.
  • D. Küçük and F. Can, “Stance detection on tweets: An svm-based approach,” arXiv preprint arXiv:1803.08910, 2018.
  • S. Ghosh et al., “Stance detection in web and social media: a comparative study,” Experimental IR Meets Multilinguality, Multimodality, and Interaction: 10th International Conference of the CLEF Association, CLEF 2019, Lugano, Switzerland, September 9–12, 2019, Proceedings 10. Springer International Publishing, 2019. L-A. Cotfas et al., “The longest month: analyzing COVID-19 vaccination opinions dynamics from tweets in the month following the first vaccine announcement,” Ieee Access, vol. 9, pp. 33203-33223, 2021. L. Grimminger and R. Klinger, “Hate towards the political opponent: A Twitter corpus study of the 2020 US elections on the basis of offensive speech and stance detection,” arXiv preprint arXiv:2103.01664, 2021.
  • K.K. Polat, N. G. Bayazıt, and O. T. Yıldız, “Türkçe duruş tespit analizi,” Avrupa Bilim ve Teknoloji Dergisi vol. 23, pp.99-107, 2021
  • D. Küçük, and N. Arıcı, “Sentiment analysis and stance detection in Turkish tweets about COVID-19 vaccination,” Handbook of research on opinion mining and text analytics on literary works and social media. IGI Global, 371-387, 2022.
  • D. Küçük, and N. Arıcı, “Deep learning-based sentiment and stance analysis of Tweets about Vaccination,” International Journal on Semantic Web and Information Systems (IJSWIS), vol. 19, no.1, pp.1-18, 2023.
  • M. S. Zengin, B. U. Yenisey, and M. Kutlu, “Exploring the impact of training datasets on Turkish stance detection,” Turkish Journal of Electrical Engineering and Computer Sciences, vol. 31, no.7, pp.1206-1222, 2023.
  • S. Arslan and E. Fırat, “Stance Detection on Short Turkish Text: A Case Study of Russia-Ukraine War,” Afyon Kocatepe Üniversitesi Fen Ve Mühendislik Bilimleri Dergisi, vol. 24, no. 3, pp. 602-619, 2024.
  • Ekşi Sözlük, “29 temmuz 2024 sokak hayvanları yasası değişikliği,” Available at https://eksisozluk.com/entry/166760652 (Accessed Date: 08.08.2024)
  • Ekşi Sözlük, “29 temmuz 2024 sokak hayvanları yasası değişikliği,” Available at https://eksisozluk.com/entry/166761830 (Accessed Date: 08.08.2024)
  • J. Devlin, M.-W. Chang, K. Lee and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
  • A. Vaswani et al., “Attention is all you need.” Advances in neural information processing systems, 30, 2017.
  • P. Savci and B. Das, “Comparison of pre-trained language models in terms of carbon emissions, time and accuracy in multi-label text classification using AutoML,” Heliyon, vol. 9, issue. 5, 2023.
  • Hugging Face, “Models,” Available at https://huggingface.co/models?other=bert (Accessed Date: 12.08.2024)
  • Z-H. Jiang et al., “Convbert: Improving bert with span-based dynamic convolution,” Advances in Neural Information Processing Systems, vol. 33, pp. 12837-12848, 2020.
  • Hugging Face, “dbmdz Turkish ConvBERT model,” Available at https://huggingface.co/dbmdz/convbert-base-turkish-cased (Accessed Date: 12.08.2024)
  • X. Chen, P. Cong and S. Lv, “A Long-Text Classification Method of Chinese News Based on BERT and CNN,” Ieee Access, vol. 10, pp. 34046-34057, 2022.
  • Medium, “Handle Long Text Corpus for Bert Model,” Available at https://medium.com/@priyatoshanand/handle-long-text-corpus-for-bert-model-3c85248214aa (Accessed Date: 13.08.2024)
  • Medium, “Fine-tuning BERT model for arbitrarily long texts, Part 1,” Available at https://medium.com/mim-solutions-blog/fine-tuning-bert-model-for-arbitrarily-long-texts-part-1-299f1533b976 (Accessed Date: 13.08.2024)
  • Ekşi Sözlük, “29 temmuz 2024 sokak hayvanları yasası değişikliği”, Available at https://eksisozluk.com/entry/166761830 (Accessed Date: 29.09.2024)
  • Y. B. Kaya and A. C. Tantuğ, “Effect of Tokenization Granularity for Turkish Large Language Models,” Journal of Intelligent Systems with Applications, vol. 21, 2024.
  • M. T. Ribeiro, S. Singh, and C. Guestrin, “ Why should I trust you?” Explaining the predictions of any classifier, In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016.
  • Ekşi Sözlük, “sahipsiz hayvanlara yönelik kanun teklifi,” Available at https://eksisozluk.com/entry/166275172 (Accessed Date: 23.09.2024)
  • Ekşi Sözlük, “sokak hayvanları uyutulacak,” Available at https://eksisozluk.com/entry/164628545 (Accessed Date: 23.09.2024)
  • Ekşi Sözlük, “sokak hayvanları uyutulacak,” Available at https://eksisozluk.com/entry/164641295 (Accessed Date: 11.01.2025)
  • Ekşi Sözlük, “sahipsiz hayvanlara yönelik kanun teklifi,” Available at https://eksisozluk.com/entry/166318051 (Accessed Date: 11.01.2025)
  • Ekşi Sözlük, “14 günde sahiplenilmeyen köpeklerin uyutulması,” Available at https://eksisozluk.com/entry/159777438 (Accessed Date: 11.01.2025)
  • Ekşi Sözlük, “sokak hayvanları uyutulacak”, Available at https://eksisozluk.com/entry/164651341 (Accessed Date: 11.01.2025)
Year 2025, Volume: 8 Issue: 1, 76 - 88, 28.03.2025
https://doi.org/10.35377/saucis...1564138

Abstract

References

  • D. Küçük and F. Can, “Stance detection: A survey,” ACM Computing Surveys (CSUR), vol. 53, no.1, pp. 1-37, 2020.
  • S. Mohammad et al., “Semeval-2016 task 6: Detecting stance in tweets,” Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016). 2016.
  • R. Cao et al., “Stance detection for online public opinion awareness: An overview,” International Journal of Intelligent Systems, vol. 37 pp. 11944-11965, 2022.
  • D. Küçük and F. Can, “Stance detection on tweets: An svm-based approach,” arXiv preprint arXiv:1803.08910, 2018.
  • S. Ghosh et al., “Stance detection in web and social media: a comparative study,” Experimental IR Meets Multilinguality, Multimodality, and Interaction: 10th International Conference of the CLEF Association, CLEF 2019, Lugano, Switzerland, September 9–12, 2019, Proceedings 10. Springer International Publishing, 2019. L-A. Cotfas et al., “The longest month: analyzing COVID-19 vaccination opinions dynamics from tweets in the month following the first vaccine announcement,” Ieee Access, vol. 9, pp. 33203-33223, 2021. L. Grimminger and R. Klinger, “Hate towards the political opponent: A Twitter corpus study of the 2020 US elections on the basis of offensive speech and stance detection,” arXiv preprint arXiv:2103.01664, 2021.
  • K.K. Polat, N. G. Bayazıt, and O. T. Yıldız, “Türkçe duruş tespit analizi,” Avrupa Bilim ve Teknoloji Dergisi vol. 23, pp.99-107, 2021
  • D. Küçük, and N. Arıcı, “Sentiment analysis and stance detection in Turkish tweets about COVID-19 vaccination,” Handbook of research on opinion mining and text analytics on literary works and social media. IGI Global, 371-387, 2022.
  • D. Küçük, and N. Arıcı, “Deep learning-based sentiment and stance analysis of Tweets about Vaccination,” International Journal on Semantic Web and Information Systems (IJSWIS), vol. 19, no.1, pp.1-18, 2023.
  • M. S. Zengin, B. U. Yenisey, and M. Kutlu, “Exploring the impact of training datasets on Turkish stance detection,” Turkish Journal of Electrical Engineering and Computer Sciences, vol. 31, no.7, pp.1206-1222, 2023.
  • S. Arslan and E. Fırat, “Stance Detection on Short Turkish Text: A Case Study of Russia-Ukraine War,” Afyon Kocatepe Üniversitesi Fen Ve Mühendislik Bilimleri Dergisi, vol. 24, no. 3, pp. 602-619, 2024.
  • Ekşi Sözlük, “29 temmuz 2024 sokak hayvanları yasası değişikliği,” Available at https://eksisozluk.com/entry/166760652 (Accessed Date: 08.08.2024)
  • Ekşi Sözlük, “29 temmuz 2024 sokak hayvanları yasası değişikliği,” Available at https://eksisozluk.com/entry/166761830 (Accessed Date: 08.08.2024)
  • J. Devlin, M.-W. Chang, K. Lee and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
  • A. Vaswani et al., “Attention is all you need.” Advances in neural information processing systems, 30, 2017.
  • P. Savci and B. Das, “Comparison of pre-trained language models in terms of carbon emissions, time and accuracy in multi-label text classification using AutoML,” Heliyon, vol. 9, issue. 5, 2023.
  • Hugging Face, “Models,” Available at https://huggingface.co/models?other=bert (Accessed Date: 12.08.2024)
  • Z-H. Jiang et al., “Convbert: Improving bert with span-based dynamic convolution,” Advances in Neural Information Processing Systems, vol. 33, pp. 12837-12848, 2020.
  • Hugging Face, “dbmdz Turkish ConvBERT model,” Available at https://huggingface.co/dbmdz/convbert-base-turkish-cased (Accessed Date: 12.08.2024)
  • X. Chen, P. Cong and S. Lv, “A Long-Text Classification Method of Chinese News Based on BERT and CNN,” Ieee Access, vol. 10, pp. 34046-34057, 2022.
  • Medium, “Handle Long Text Corpus for Bert Model,” Available at https://medium.com/@priyatoshanand/handle-long-text-corpus-for-bert-model-3c85248214aa (Accessed Date: 13.08.2024)
  • Medium, “Fine-tuning BERT model for arbitrarily long texts, Part 1,” Available at https://medium.com/mim-solutions-blog/fine-tuning-bert-model-for-arbitrarily-long-texts-part-1-299f1533b976 (Accessed Date: 13.08.2024)
  • Ekşi Sözlük, “29 temmuz 2024 sokak hayvanları yasası değişikliği”, Available at https://eksisozluk.com/entry/166761830 (Accessed Date: 29.09.2024)
  • Y. B. Kaya and A. C. Tantuğ, “Effect of Tokenization Granularity for Turkish Large Language Models,” Journal of Intelligent Systems with Applications, vol. 21, 2024.
  • M. T. Ribeiro, S. Singh, and C. Guestrin, “ Why should I trust you?” Explaining the predictions of any classifier, In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016.
  • Ekşi Sözlük, “sahipsiz hayvanlara yönelik kanun teklifi,” Available at https://eksisozluk.com/entry/166275172 (Accessed Date: 23.09.2024)
  • Ekşi Sözlük, “sokak hayvanları uyutulacak,” Available at https://eksisozluk.com/entry/164628545 (Accessed Date: 23.09.2024)
  • Ekşi Sözlük, “sokak hayvanları uyutulacak,” Available at https://eksisozluk.com/entry/164641295 (Accessed Date: 11.01.2025)
  • Ekşi Sözlük, “sahipsiz hayvanlara yönelik kanun teklifi,” Available at https://eksisozluk.com/entry/166318051 (Accessed Date: 11.01.2025)
  • Ekşi Sözlük, “14 günde sahiplenilmeyen köpeklerin uyutulması,” Available at https://eksisozluk.com/entry/159777438 (Accessed Date: 11.01.2025)
  • Ekşi Sözlük, “sokak hayvanları uyutulacak”, Available at https://eksisozluk.com/entry/164651341 (Accessed Date: 11.01.2025)
There are 30 citations in total.

Details

Primary Language English
Subjects Software Engineering (Other)
Journal Section Research Article
Authors

Selma Alav 0009-0009-1521-032X

Kristin Surpuhi Benli 0000-0001-6282-6703

Early Pub Date March 27, 2025
Publication Date March 28, 2025
Submission Date October 9, 2024
Acceptance Date March 5, 2025
Published in Issue Year 2025Volume: 8 Issue: 1

Cite

APA Alav, S., & Benli, K. S. (2025). Turkish Stance Detection on Social Media Using BERT Models: A Case Study of Stray Animals Law. Sakarya University Journal of Computer and Information Sciences, 8(1), 76-88. https://doi.org/10.35377/saucis...1564138
AMA Alav S, Benli KS. Turkish Stance Detection on Social Media Using BERT Models: A Case Study of Stray Animals Law. SAUCIS. March 2025;8(1):76-88. doi:10.35377/saucis.1564138
Chicago Alav, Selma, and Kristin Surpuhi Benli. “Turkish Stance Detection on Social Media Using BERT Models: A Case Study of Stray Animals Law”. Sakarya University Journal of Computer and Information Sciences 8, no. 1 (March 2025): 76-88. https://doi.org/10.35377/saucis. 1564138.
EndNote Alav S, Benli KS (March 1, 2025) Turkish Stance Detection on Social Media Using BERT Models: A Case Study of Stray Animals Law. Sakarya University Journal of Computer and Information Sciences 8 1 76–88.
IEEE S. Alav and K. S. Benli, “Turkish Stance Detection on Social Media Using BERT Models: A Case Study of Stray Animals Law”, SAUCIS, vol. 8, no. 1, pp. 76–88, 2025, doi: 10.35377/saucis...1564138.
ISNAD Alav, Selma - Benli, Kristin Surpuhi. “Turkish Stance Detection on Social Media Using BERT Models: A Case Study of Stray Animals Law”. Sakarya University Journal of Computer and Information Sciences 8/1 (March 2025), 76-88. https://doi.org/10.35377/saucis. 1564138.
JAMA Alav S, Benli KS. Turkish Stance Detection on Social Media Using BERT Models: A Case Study of Stray Animals Law. SAUCIS. 2025;8:76–88.
MLA Alav, Selma and Kristin Surpuhi Benli. “Turkish Stance Detection on Social Media Using BERT Models: A Case Study of Stray Animals Law”. Sakarya University Journal of Computer and Information Sciences, vol. 8, no. 1, 2025, pp. 76-88, doi:10.35377/saucis. 1564138.
Vancouver Alav S, Benli KS. Turkish Stance Detection on Social Media Using BERT Models: A Case Study of Stray Animals Law. SAUCIS. 2025;8(1):76-88.


INDEXING & ABSTRACTING & ARCHIVING


 31045 31044  31046 31047 

31043 28939 28938


29070    The papers in this journal are licensed under a Creative Commons Attribution-NonCommercial 4.0 International License