TurkishLex:Development of a Context-Aware Spell Checker for Detecting and Correcting Spelling Errors in Turkish Texts

Pinar Savci; Bihter Daş

doi:10.35377/saucis.7.87942.1544012

EN

TurkishLex:Development of a Context-Aware Spell Checker for Detecting and Correcting Spelling Errors in Turkish Texts

Abstract

In Turkish, correct spelling correction is crucial for effective communication and preserving the integrity of written text. The challenge lies in the complexity of Turkish morphology and spelling, which can lead to frequent and diverse spelling errors. This study presents a spelling checker adapted for Turkish by creating a new Turkish dataset. The proposed spelling checker model effectively captures both minor and major textual changes and can detect the error. Our findings show that the proposed spelling checker system provides high accuracy and reliability with 98.21% accuracy performance with the Symspell module in correcting Turkish texts. This study provides valuable information about the strengths and weaknesses of existing spelling checkers and contributes to the improvement of spelling correction tools for Turkish.

Keywords

Project Number

AR-22-087-0001

References

Y. Chaabi and F. Ataa Allah, “Amazigh spell checker using Damerau-Levenshtein algorithm and N-gram,” Journal of King Saud University - Computer and Information Sciences, vol. 34, no. 8, Part B, pp. 6116–6124, Sep. 2022, doi: 10.1016/j.jksuci.2021.07.015.
V. J. Hodge and J. Austin, “A comparison of a novel neural spell checker and standard spell checking algorithms,” Pattern Recognition, vol. 35, no. 11, pp. 2571–2580, Nov. 2002, doi: 10.1016/S0031-3203(01)00174-1.
R. Garfinkel, E. Fernandez, and R. Gopal, “Design of an interactive spell checker: Optimizing the list of offered words,” Decision Support Systems, vol. 35, no. 3, pp. 385–397, Jun. 2003, doi: 10.1016/S0167-9236(02)00115-X.
M. Nejja and A. Yousfi, “The Context in Automatic Spell Correction,” Procedia Computer Science, vol. 73, pp. 109–114, Jan. 2015, doi: 10.1016/j.procs.2015.12.055.
K. Sarıtaş, C. A. Öz, and T. Güngör, “A comprehensive analysis of static word embeddings for Turkish,” Expert Systems with Applications, vol. 252, p. 124123, Oct. 2024, doi: 10.1016/j.eswa.2024.124123.
S. Demir and B. Topcu, “Graph-based Turkish text normalization and its impact on noisy text processing,” Engineering Science and Technology, an International Journal, vol. 35, p. 101192, Nov. 2022, doi: 10.1016/j.jestch.2022.101192.
Y. B. Kaya and A. C. Tantuğ, “Effect of tokenization granularity for Turkish large language models,” Intelligent Systems with Applications, vol. 21, p. 200335, Mar. 2024, doi: 10.1016/j.iswa.2024.200335.
Kukich K. Techniques for automatically correcting words in text. ACM computing surveys (CSUR). 1992 Dec 1;24(4):377-439.

P. T. Hacken and C. Tschichold, “Word Manager and CALL: Structured access to the lexicon as a tool for enriching learners’ vocabulary,” ReCALL, vol. 13, no. 1, pp. 121–131, May 2001, doi: 10.1017/S0958344001001112.
W. Phatthiyaphaibun, K. Chaovavanich, C. Polpanumas, A. Suriyawongkul, L. Lowphansirikul, and P. Chormai, PyThaiNLP: Thai Natural Language Processing in Python. (Jun. 2024). Python. Accessed: Aug. 27, 2024. [Online]. Available: https://github.com/PyThaiNLP/pythainlp
hunspell/hunspell. (Aug. 27, 2024). C++. hunspell. Accessed: Aug. 27, 2024. [Online]. Available: https://github.com/hunspell/hunspell
A. Lertpiya, T. Chaiwachirasak, N. Maharattanamalai, T. Lapjaturapit, T. Chalothorn, N. Tirasaroj, et al., "A preliminary study on fundamental Thai NLP tasks for user-generated Web content", Proc. Int. Joint Symp. Artif. Intell. Natural Lang. Process. (iSAI-NLP), pp. 1-8, Nov. 2018.
S. Watcharabutsarakham, "Spell checker for Thai document", Proc. IEEE Region Conf., pp. 1-4, Nov. 2005.
M. Rodphon, K. Siriboon and B. Kruatrachue, "Thai OCR error correction using token passing algorithm", Proc. IEEE Pacific Rim Conf. Commun. Comput. Signal Process., pp. 599-602, 2001.
B. Kruatrachue, K. Somguntar and K. Siriboon, "Thai OCR error correction using genetic algorithm", Proc. 1st Int. Symp. Cyber Worlds, pp. 137-141, 2002.
H. T. Ng, S. M. Wu, T. Briscoe, C. Hadiwinoto, R. H. Susanto and C. Bryant, "The CoNLL-2014 shared task on grammatical error correction", Proc. 18th Conf. Comput. Natural Lang. Learn. Shared Task, pp. 1-14, 2014.
A. Rozovskaya and D. Roth, "Grammatical error correction: Machine translation and classifiers", Proc. 54th Annu. Meeting Assoc. Comput. Linguistic, pp. 2205-2215, Aug. 2016, [online] Available: https://www.aclweb.org/anthology/P16-1208.
M. Junczys-Dowmunt and R. Grundkiewicz, "Phrase-based Machine Translation is State-of-the-Art for Automatic Grammatical Error Correction", Proc. Conf. Empirical Methods Natural Lang. Process., pp. 1546-1556, Nov. 2016, [online] Available: https://www.aclweb.org/anthology/D16-1161.
Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (pp. 5998-6008).
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692.
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative adversarial nets. In Advances in Neural Information Processing Systems (pp. 2672-2680).
D. N. Mati, M. Hamiti, B. Selimi and J. Ajdari, "Building Spell-Check Dictionary for Low-Resource Language by Comparing Word Usage," 2021 44th International Convention on Information, Communication and Electronic Technology (MIPRO), Opatija, Croatia, 2021, pp. 229-236, doi: 10.23919/MIPRO52101.2021.9597183.
A. Kicsi, K. Szabó Ledenyi, and L. Vidács, “Radiologic text correction for better machine understanding,” Engineering Reports, vol. n/a, no. n/a, p. e12891, doi: 10.1002/eng2.12891.
D. Pogrebnoi, A. Funkner, and S. Kovalchuk, “RuMedSpellchecker: A new approach for advanced spelling error correction in Russian electronic health records,” Journal of Computational Science, vol. 82, p. 102393, Oct. 2024, doi: 10.1016/j.jocs.2024.102393.
E., O’Neill, R., Young, E., Thiaville, M., MacCarthy, J., Carson-Berndsen, & A. Ventresque, S-capade: Spelling correction aimed at particularly deviant errors. In Statistical Language and Speech Processing: 8th International Conference, SLSP 2020, Cardiff, UK, October 14–16, 2020, Proceedings 8 (pp. 85-96). Springer International Publishing.
U., Liyanapathirana, K., Gunasinghe, & G. Dias. Sinspell: A comprehensive spelling checker for sinhala. arXiv preprint arXiv:2107.02983, 2021.
O. Abiola, A. Abayomi-Alli, O. A. Tale, S. Misra, and O. Abayomi-Alli, “Sentiment analysis of COVID-19 tweets from selected hashtags in Nigeria using VADER and Text Blob analyser,” Journal of Electrical Systems and Inf Technol, vol. 10, no. 1, p. 5, Jan. 2023, doi: 10.1186/s43067-023-00070-9.
N. Bölücü and B. Can, "Context Based Automatic Spelling Correction for Turkish," 2019 Scientific Meeting on Electrical-Electronics & Biomedical Engineering and Computer Science (EBBT), Istanbul, Turkey, 2019, pp. 1-4, doi: 10.1109/EBBT.2019.8742067.
Aydoğan, M., & Karci, A. (2020). Spelling Correction with the Dictionary Method for the Turkish Language Using Word Embeddings. Avrupa Bilim ve Teknoloji Dergisi, 57–63. https://doi.org/10.31590/ejosat.araconf8
O. Büyük, M. Erden and L. M. Arslan, "Context Influence on Sequence to Sequence Turkish Spelling Correction," 2019 27th Signal Processing and Communications Applications Conference (SIU), Sivas, Turkey, 2019, pp. 1-4, doi: 10.1109/SIU.2019.8806476.

Details

Primary Language

English

Subjects

Software Engineering (Other)

Journal Section

Research Article

Authors

Pinar Savci
Türkiye

Bihter Daş ^*
0000-0002-2498-3297
Türkiye

Early Pub Date

December 10, 2024

Publication Date

December 31, 2024

Submission Date

September 5, 2024

Acceptance Date

October 10, 2024

Published in Issue

Year 2024 Volume: 7 Number: 3

DOI

https://doi.org/10.35377/saucis.7.87942.1544012

IZ

https://izlik.org/JA75SD32PY

Cite

RIS / Bibtex

APA

Savci, P., & Daş, B. (2024). TurkishLex:Development of a Context-Aware Spell Checker for Detecting and Correcting Spelling Errors in Turkish Texts. Sakarya University Journal of Computer and Information Sciences, 7(3), 404-415. https://doi.org/10.35377/saucis.7.87942.1544012

AMA

1.Savci P, Daş B. TurkishLex:Development of a Context-Aware Spell Checker for Detecting and Correcting Spelling Errors in Turkish Texts. SAUCIS. 2024;7(3):404-415. doi:10.35377/saucis.7.87942.1544012

Chicago

Savci, Pinar, and Bihter Daş. 2024. “TurkishLex:Development of a Context-Aware Spell Checker for Detecting and Correcting Spelling Errors in Turkish Texts”. Sakarya University Journal of Computer and Information Sciences 7 (3): 404-15. https://doi.org/10.35377/saucis.7.87942.1544012.

EndNote

Savci P, Daş B (December 1, 2024) TurkishLex:Development of a Context-Aware Spell Checker for Detecting and Correcting Spelling Errors in Turkish Texts. Sakarya University Journal of Computer and Information Sciences 7 3 404–415.

IEEE

[1]P. Savci and B. Daş, “TurkishLex:Development of a Context-Aware Spell Checker for Detecting and Correcting Spelling Errors in Turkish Texts”, SAUCIS, vol. 7, no. 3, pp. 404–415, Dec. 2024, doi: 10.35377/saucis.7.87942.1544012.

ISNAD

Savci, Pinar - Daş, Bihter. “TurkishLex:Development of a Context-Aware Spell Checker for Detecting and Correcting Spelling Errors in Turkish Texts”. Sakarya University Journal of Computer and Information Sciences 7/3 (December 1, 2024): 404-415. https://doi.org/10.35377/saucis.7.87942.1544012.

JAMA

1.Savci P, Daş B. TurkishLex:Development of a Context-Aware Spell Checker for Detecting and Correcting Spelling Errors in Turkish Texts. SAUCIS. 2024;7:404–415.

MLA

Savci, Pinar, and Bihter Daş. “TurkishLex:Development of a Context-Aware Spell Checker for Detecting and Correcting Spelling Errors in Turkish Texts”. Sakarya University Journal of Computer and Information Sciences, vol. 7, no. 3, Dec. 2024, pp. 404-15, doi:10.35377/saucis.7.87942.1544012.

Vancouver

1.Pinar Savci, Bihter Daş. TurkishLex:Development of a Context-Aware Spell Checker for Detecting and Correcting Spelling Errors in Turkish Texts. SAUCIS. 2024 Dec. 1;7(3):404-15. doi:10.35377/saucis.7.87942.1544012