Research Article
BibTex RIS Cite

Customer Dissatisfaction Towards Chatbot Services of e-Commerce Shopping Sites: A Qualitative Analysis

Year 2024, Volume: 2024 Issue: Erken Görünüm

Burak Can Altay Naim Çetintürk

https://doi.org/10.26650/JTL.2024.1355850

Abstract

This research examines the customers’ comments about the chatbots published in a customer complaint website while shopping from an e-commerce site or app. First, 89 customers’ complaints were imported from a customer complaint platform to a single document. Then, the document was subjected to content analysis using a qualitative research tool, Maxqda Plus 2022, and each comment was categorized under related complaint categories. Second, the frequency of customer complaints categories was calculated using the same tool. Additionally, visual maps for each category were created to make the complaints more understandable. While these categorical variables have been addressed in previous studies, variables based on consumer feedback have only been included in this study. According to the research findings, the most frequent customer complaint category is about meaningfulness (with a share of 47.6% in the general total). The least frequent ones are the inability to find a real contact person and the absence of chatbot service (with a share of 7.9%).

Keywords

E-commerce Chatbot Customer experience Customer satisfaction Dissatisfaction

References

  • Acumen Research and Consulting. (2022). Access: 09.01.2022, https://www.globenewswire.com/news-release/2022/07/25/2485463/0/en/Chatbot-Market-Size-to-Grow-to-USD-3-411-Million-by-2030-Propelled-By-the-Growing-Use-of-Bots-for-Marketing-and-Promotion-Activities.html google scholar
  • Almansor, E. H., Hussan, F. K. & Hussain, O. K. (2021). Supervised ensemble sentiment-based framework to measure chatbot quality of services. Computing, 103(3), 491-507. google scholar
  • Anantaram, C. & Sangroya, A. (2017). Identifying latent beliefs in customer complaints to trigger epistemic rules for relevant human-bot dialog. 3rd International Conference on Control, Automation and Robotics (ICCAR), 731-734. google scholar
  • Arya, M. (2019). Access: 09.01.2022, https://chatbotslife.com/a-brief-history-of-chatbots-d5a8689cf52f google scholar
  • Behere, T., Vaidya, A., Birhade, A., Shinde, K., Deshpande, P. & Jahirabadkar, S. (2020). Text summarization and classification of conversation data between service chatbot and customer. 2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4), 833-838. google scholar
  • Brandtzaeg, P. B. & F0lstad, A. (2017). Why people use chatbots. International Conference on Internet Science, 377-392. google scholar
  • Bührke, J., Brendel, A. B., Lichtenberg, S., Greve, M. & Mirbabaie, M. (2021). Is making mistakes human? On the perception of typing errors in chatbot communication. In Proceedings of the 54th Hawaii International Conference on System Sciences. google scholar
  • Chaves, A. P & Gerosa, M. A. (2020). How should my chatbot interact? A survey on social characteristics in human-chatbot interaction design. International Journal of Human-Computer interaction. google scholar
  • Cheng, H.-T. & Pan, Y. (2021). "I’m not a Chatbot": An empirical investigation of humanized profiles of social media customer service representatives. 54th Hawaii International Conference on System Sciences, 4167-4176. google scholar
  • Chung, M., Ko, E., Joung, H. & Kim, S. J. (2020). Chatbot e-service and customer satisfaction regarding luxury brands. Journal of Business Research, 117, 587-595. google scholar
  • Collins, C., Arbour, S., B. N., Yama, S., Laffier, J. & Zhao, Z. (2021). Covid connect: Chat-driven anonymous story-sharing for peer support. Designing Interactive Systems Conference, 301-318. google scholar
  • Crolic, C., Thomaz, F., Hadi, R. & Stephen, A. T. (2022). Blame the bot: Anthropomorphism and anger in customer-chatbot interactions. Journal of Marketing, 132-148. google scholar
  • Cui, L., Huang, S., Wei, F., Tan, C., Duan, C. & Zhou, M. (2017). Superagent: A customer service chatbot for e-commerce websites. Proceedings of ACL 2017, 97-102. google scholar
  • Davenport, T. H. & Klahr, P. (1998). Managing customer support knowledge. California Management Review, 40(3), 195-208. google scholar
  • Davidow, M. (2003). Organizational responses to customer complaints: What works and what doesn’t. Journal of Service Research, 5(3), 225-250. google scholar
  • Dosovitsky, G. & Bunge, E. L. (2021). Bonding with bot: User feedback on a chatbot for social isolation. Frontiers in Digital Health, 3, 1-11. google scholar
  • Duan, Y., Yoon, M., Liang, Z. & Hoorn, J. F. (2021). Self-disclosure to a robot: only for those who suffer the most. Robotics, 10(3). google scholar
  • Federici, S., Filippis, M. L., Mele, M. L., Borsci, S., Bracalenti, M., Gaudino, G. & Simonetti, E. (2020). Inside pandora’s box: a systematic review of the assessment of the perceived quality of chatbots for people with disabilities or special needs. Disability and Rehabilitation: Assistive Technology, 15(7), 832-837. google scholar
  • F0lstad, A., Nordheim, C. B., & Bj0rkli, C. A. (2018). What makes users trust a chatbot for customer service? An exploratory interview study. International Conference on Internet Science, 194-208. google scholar
  • Gashi, F., Regli, S. F., May, R., Tschopp, P. & Denecke, K. (2021). Developing intelligent interviewers to collect the medical history: Lessons learned and guidelines. dHealth, 18-25. google scholar
  • Harrison-Walker, L. J. (2001). E-complaining: A content analysis of an internet complaint forum. Journal of Services Marketing, 15(5), s. 397-412. google scholar
  • Hill, J., Ford, W. R. & Farreras, I. G. (2015). Real conversations with artificial intelligence: A comparison between human-human online conversations and human-chatbot conversations. Computers in Human Behavior, 49, 245-250. google scholar
  • Huang, J., Zhou, M. & Yang, D. (2007). Extracting chatbot knowledge from online discussion forums. IJCAI, 423-428. google scholar
  • Jayachandran, S., Hewett, K. & Kaufman. (2004). Customer response capability in a sense-and-respond era: the role of customer knowledge process. Journal of the Academy of Marketing Science, 32(3), 219-233. google scholar
  • Jenkins, M. C., Churchill, R., Cox, S. & Smith, D. (2007). Analysis of user interaction with service oriented chatbot systems. International Conference On Human-Computer Interaction, 76-83. google scholar
  • Khan, S. & Rabbani, M. R. (2020). Chatbot as Islamic finance expert (CaIFE): When finance meets artificial intellige. Proceedings of the 2020 4th International Symposium on Computer Science and Intelligent Control, 1-5. google scholar
  • Kuberkar, S. & Singhal, T. K. (2020). Factors influencing adoption intention of A.I. powered chatbot for public transport services within a smart city. International Journal of Emerging Technologies in Learning, 11(3), 948-958. google scholar
  • Liu, C., Zhou, S., Zhang, Y., Liu, D., Peng, Z. & Ma, X. (2022). Exploring the effects of self-mockery to improve task-oriented chatbot’s social intelligence. Designing Interactive Systems Conference, 1315-1329. google scholar
  • Narasiman, S. K., Srinivassababu, T. H., Suhit Raja, S. & Babu, R. (2019). IndQuery-An online portal for registering e-complaints integrated with smart chatbot. International Conference on Emerging Current Trends in Computing and Expert Technology, 1286-1294. google scholar
  • Narendra, L. W. & Setyaningsih, E. R. (2021). Designing a transactional smart assistant in indonesian using rasa framework. 7th International Conference on Electrical, Electronics and Information Engineering, 1-6. google scholar
  • Narynov, S., Zhumanov, Z., Gumar, A., K.M. & Omarov, B. (2021). Development of chatbot psychologist applying natural language understanding techniques. 21st International Conference on Control, Automation and Systems (ICCAS), 636-641. google scholar
  • Ni, L., Lu, C., Liu, N. & Liu, J. (2017). Mandy: Towards a smart primary care chatbot application. International symposium on knowledge and systems sciences, 38-52. google scholar
  • Orellana, C., Tobar, M. Y., J., P.-O. D. & Guachi-Guachi, L. (2021). A chatterbot based on genetic algorithm: Preliminary results. International Conference on Applied Informatics , 3-12. google scholar
  • Othlinghaus-Wulhorst, J. & Hoppe, H. U. (2020). A technical and conceptual framework for serious role-playing games in the area of social skill training. Frontiers in Computer Science, 2(28), 1-20. google scholar
  • Raundale, P. & Sawale, A. (2021). Dialog prediction in institute admission: A deep learning way. 2nd International Conference for Emerging Technology (INCET), 1-5. google scholar
  • Rigamonti, L., Estel, K., Gehlen, T., Wolfarth, B., Lawrence, J. B. & Back, D. A. (2021). Use of artificial intelligence in sports medicine: a report of5 fictional cases. BMC Sports Science, Medicine and Rehabilitation, 13(1), 1-7. google scholar
  • Sangroya, A., Anantaram, C., Saini, P. & Rawat, M. (2018). Extracting latent beliefs and using epistemic reasoning to tailor a chatbot. JCAI, 5853-5855. google scholar
  • Shawar, B. A. & Atwell, E. (2007). Different measurements metrics to evaluate a chatbot system. Proceedings of the Workshop on Bridging the Gap: Academic And Industrial Research in Dialog Technologies, 89-96. google scholar
  • Suhaili, S. M., Salim, N. & Jambli, M. N. (2021). Service chatbots: A systematic review. Expert Systems with Applications, 184. google scholar
  • Suresh, N., Mukabe, N., Hashiyana, V., Limbo, A. & Hauwanga, A. (2021). Career counseling chatbot on facebook messenger using A.I. ACM International Conference Proceeding Series, 65-73. google scholar
  • Temple, J. G. & Elie, C. J. (2019). Beyond the chatbot: enhancing search with cognitive capabilities. International Conference on Applied Human Factors and Ergonomics, 283-290. google scholar
  • Van Den Broeck, E., Zarouali, B. & Poels, K. (2019). Chatbot advertising effectiveness: When does the message get through? Computers in Human Behavior, 98, 150-157. google scholar
  • Waghmare, C. (2019). Business Benefits of Using Chatbots. Apress. google scholar
  • Xu, A., Liu, Z., Guo, Y., Sinha, V. & Akkiraju, R. (2017). A new chatbot for customer service on social media. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 3506-3510. google scholar
  • Zamora, J. (2017). I’m sorry, dave, i’m afraid i can’t do that: Chatbot perception and expectations. Proceedings of the 5th International Conference on Human Agent Interaction, 253-260. google scholar

Year 2024, Volume: 2024 Issue: Erken Görünüm

Burak Can Altay Naim Çetintürk

https://doi.org/10.26650/JTL.2024.1355850

Abstract

References

  • Acumen Research and Consulting. (2022). Access: 09.01.2022, https://www.globenewswire.com/news-release/2022/07/25/2485463/0/en/Chatbot-Market-Size-to-Grow-to-USD-3-411-Million-by-2030-Propelled-By-the-Growing-Use-of-Bots-for-Marketing-and-Promotion-Activities.html google scholar
  • Almansor, E. H., Hussan, F. K. & Hussain, O. K. (2021). Supervised ensemble sentiment-based framework to measure chatbot quality of services. Computing, 103(3), 491-507. google scholar
  • Anantaram, C. & Sangroya, A. (2017). Identifying latent beliefs in customer complaints to trigger epistemic rules for relevant human-bot dialog. 3rd International Conference on Control, Automation and Robotics (ICCAR), 731-734. google scholar
  • Arya, M. (2019). Access: 09.01.2022, https://chatbotslife.com/a-brief-history-of-chatbots-d5a8689cf52f google scholar
  • Behere, T., Vaidya, A., Birhade, A., Shinde, K., Deshpande, P. & Jahirabadkar, S. (2020). Text summarization and classification of conversation data between service chatbot and customer. 2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4), 833-838. google scholar
  • Brandtzaeg, P. B. & F0lstad, A. (2017). Why people use chatbots. International Conference on Internet Science, 377-392. google scholar
  • Bührke, J., Brendel, A. B., Lichtenberg, S., Greve, M. & Mirbabaie, M. (2021). Is making mistakes human? On the perception of typing errors in chatbot communication. In Proceedings of the 54th Hawaii International Conference on System Sciences. google scholar
  • Chaves, A. P & Gerosa, M. A. (2020). How should my chatbot interact? A survey on social characteristics in human-chatbot interaction design. International Journal of Human-Computer interaction. google scholar
  • Cheng, H.-T. & Pan, Y. (2021). "I’m not a Chatbot": An empirical investigation of humanized profiles of social media customer service representatives. 54th Hawaii International Conference on System Sciences, 4167-4176. google scholar
  • Chung, M., Ko, E., Joung, H. & Kim, S. J. (2020). Chatbot e-service and customer satisfaction regarding luxury brands. Journal of Business Research, 117, 587-595. google scholar
  • Collins, C., Arbour, S., B. N., Yama, S., Laffier, J. & Zhao, Z. (2021). Covid connect: Chat-driven anonymous story-sharing for peer support. Designing Interactive Systems Conference, 301-318. google scholar
  • Crolic, C., Thomaz, F., Hadi, R. & Stephen, A. T. (2022). Blame the bot: Anthropomorphism and anger in customer-chatbot interactions. Journal of Marketing, 132-148. google scholar
  • Cui, L., Huang, S., Wei, F., Tan, C., Duan, C. & Zhou, M. (2017). Superagent: A customer service chatbot for e-commerce websites. Proceedings of ACL 2017, 97-102. google scholar
  • Davenport, T. H. & Klahr, P. (1998). Managing customer support knowledge. California Management Review, 40(3), 195-208. google scholar
  • Davidow, M. (2003). Organizational responses to customer complaints: What works and what doesn’t. Journal of Service Research, 5(3), 225-250. google scholar
  • Dosovitsky, G. & Bunge, E. L. (2021). Bonding with bot: User feedback on a chatbot for social isolation. Frontiers in Digital Health, 3, 1-11. google scholar
  • Duan, Y., Yoon, M., Liang, Z. & Hoorn, J. F. (2021). Self-disclosure to a robot: only for those who suffer the most. Robotics, 10(3). google scholar
  • Federici, S., Filippis, M. L., Mele, M. L., Borsci, S., Bracalenti, M., Gaudino, G. & Simonetti, E. (2020). Inside pandora’s box: a systematic review of the assessment of the perceived quality of chatbots for people with disabilities or special needs. Disability and Rehabilitation: Assistive Technology, 15(7), 832-837. google scholar
  • F0lstad, A., Nordheim, C. B., & Bj0rkli, C. A. (2018). What makes users trust a chatbot for customer service? An exploratory interview study. International Conference on Internet Science, 194-208. google scholar
  • Gashi, F., Regli, S. F., May, R., Tschopp, P. & Denecke, K. (2021). Developing intelligent interviewers to collect the medical history: Lessons learned and guidelines. dHealth, 18-25. google scholar
  • Harrison-Walker, L. J. (2001). E-complaining: A content analysis of an internet complaint forum. Journal of Services Marketing, 15(5), s. 397-412. google scholar
  • Hill, J., Ford, W. R. & Farreras, I. G. (2015). Real conversations with artificial intelligence: A comparison between human-human online conversations and human-chatbot conversations. Computers in Human Behavior, 49, 245-250. google scholar
  • Huang, J., Zhou, M. & Yang, D. (2007). Extracting chatbot knowledge from online discussion forums. IJCAI, 423-428. google scholar
  • Jayachandran, S., Hewett, K. & Kaufman. (2004). Customer response capability in a sense-and-respond era: the role of customer knowledge process. Journal of the Academy of Marketing Science, 32(3), 219-233. google scholar
  • Jenkins, M. C., Churchill, R., Cox, S. & Smith, D. (2007). Analysis of user interaction with service oriented chatbot systems. International Conference On Human-Computer Interaction, 76-83. google scholar
  • Khan, S. & Rabbani, M. R. (2020). Chatbot as Islamic finance expert (CaIFE): When finance meets artificial intellige. Proceedings of the 2020 4th International Symposium on Computer Science and Intelligent Control, 1-5. google scholar
  • Kuberkar, S. & Singhal, T. K. (2020). Factors influencing adoption intention of A.I. powered chatbot for public transport services within a smart city. International Journal of Emerging Technologies in Learning, 11(3), 948-958. google scholar
  • Liu, C., Zhou, S., Zhang, Y., Liu, D., Peng, Z. & Ma, X. (2022). Exploring the effects of self-mockery to improve task-oriented chatbot’s social intelligence. Designing Interactive Systems Conference, 1315-1329. google scholar
  • Narasiman, S. K., Srinivassababu, T. H., Suhit Raja, S. & Babu, R. (2019). IndQuery-An online portal for registering e-complaints integrated with smart chatbot. International Conference on Emerging Current Trends in Computing and Expert Technology, 1286-1294. google scholar
  • Narendra, L. W. & Setyaningsih, E. R. (2021). Designing a transactional smart assistant in indonesian using rasa framework. 7th International Conference on Electrical, Electronics and Information Engineering, 1-6. google scholar
  • Narynov, S., Zhumanov, Z., Gumar, A., K.M. & Omarov, B. (2021). Development of chatbot psychologist applying natural language understanding techniques. 21st International Conference on Control, Automation and Systems (ICCAS), 636-641. google scholar
  • Ni, L., Lu, C., Liu, N. & Liu, J. (2017). Mandy: Towards a smart primary care chatbot application. International symposium on knowledge and systems sciences, 38-52. google scholar
  • Orellana, C., Tobar, M. Y., J., P.-O. D. & Guachi-Guachi, L. (2021). A chatterbot based on genetic algorithm: Preliminary results. International Conference on Applied Informatics , 3-12. google scholar
  • Othlinghaus-Wulhorst, J. & Hoppe, H. U. (2020). A technical and conceptual framework for serious role-playing games in the area of social skill training. Frontiers in Computer Science, 2(28), 1-20. google scholar
  • Raundale, P. & Sawale, A. (2021). Dialog prediction in institute admission: A deep learning way. 2nd International Conference for Emerging Technology (INCET), 1-5. google scholar
  • Rigamonti, L., Estel, K., Gehlen, T., Wolfarth, B., Lawrence, J. B. & Back, D. A. (2021). Use of artificial intelligence in sports medicine: a report of5 fictional cases. BMC Sports Science, Medicine and Rehabilitation, 13(1), 1-7. google scholar
  • Sangroya, A., Anantaram, C., Saini, P. & Rawat, M. (2018). Extracting latent beliefs and using epistemic reasoning to tailor a chatbot. JCAI, 5853-5855. google scholar
  • Shawar, B. A. & Atwell, E. (2007). Different measurements metrics to evaluate a chatbot system. Proceedings of the Workshop on Bridging the Gap: Academic And Industrial Research in Dialog Technologies, 89-96. google scholar
  • Suhaili, S. M., Salim, N. & Jambli, M. N. (2021). Service chatbots: A systematic review. Expert Systems with Applications, 184. google scholar
  • Suresh, N., Mukabe, N., Hashiyana, V., Limbo, A. & Hauwanga, A. (2021). Career counseling chatbot on facebook messenger using A.I. ACM International Conference Proceeding Series, 65-73. google scholar
  • Temple, J. G. & Elie, C. J. (2019). Beyond the chatbot: enhancing search with cognitive capabilities. International Conference on Applied Human Factors and Ergonomics, 283-290. google scholar
  • Van Den Broeck, E., Zarouali, B. & Poels, K. (2019). Chatbot advertising effectiveness: When does the message get through? Computers in Human Behavior, 98, 150-157. google scholar
  • Waghmare, C. (2019). Business Benefits of Using Chatbots. Apress. google scholar
  • Xu, A., Liu, Z., Guo, Y., Sinha, V. & Akkiraju, R. (2017). A new chatbot for customer service on social media. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 3506-3510. google scholar
  • Zamora, J. (2017). I’m sorry, dave, i’m afraid i can’t do that: Chatbot perception and expectations. Proceedings of the 5th International Conference on Human Agent Interaction, 253-260. google scholar
There are 45 citations in total.

Details

Primary Language English
Subjects Transportation, Logistics and Supply Chains (Other)
Journal Section Research Article
Authors

Burak Can Altay 0000-0002-0572-8848

Naim Çetintürk 0000-0002-8681-320X

Early Pub Date March 22, 2024
Publication Date
Submission Date September 8, 2023
Acceptance Date December 15, 2023
Published in Issue Year 2024 Volume: 2024 Issue: Erken Görünüm

Cite

APA Altay, B. C., & Çetintürk, N. (2024). Customer Dissatisfaction Towards Chatbot Services of e-Commerce Shopping Sites: A Qualitative Analysis. Journal of Transportation and Logistics, 2024(Erken Görünüm). https://doi.org/10.26650/JTL.2024.1355850
AMA Altay BC, Çetintürk N. Customer Dissatisfaction Towards Chatbot Services of e-Commerce Shopping Sites: A Qualitative Analysis. JTL. March 2024;2024(Erken Görünüm). doi:10.26650/JTL.2024.1355850
Chicago Altay, Burak Can, and Naim Çetintürk. “Customer Dissatisfaction Towards Chatbot Services of E-Commerce Shopping Sites: A Qualitative Analysis”. Journal of Transportation and Logistics 2024, no. Erken Görünüm (March 2024). https://doi.org/10.26650/JTL.2024.1355850.
EndNote Altay BC, Çetintürk N (March 1, 2024) Customer Dissatisfaction Towards Chatbot Services of e-Commerce Shopping Sites: A Qualitative Analysis. Journal of Transportation and Logistics 2024 Erken Görünüm
IEEE B. C. Altay and N. Çetintürk, “Customer Dissatisfaction Towards Chatbot Services of e-Commerce Shopping Sites: A Qualitative Analysis”, JTL, vol. 2024, no. Erken Görünüm, 2024, doi: 10.26650/JTL.2024.1355850.
ISNAD Altay, Burak Can - Çetintürk, Naim. “Customer Dissatisfaction Towards Chatbot Services of E-Commerce Shopping Sites: A Qualitative Analysis”. Journal of Transportation and Logistics 2024/Erken Görünüm (March 2024). https://doi.org/10.26650/JTL.2024.1355850.
JAMA Altay BC, Çetintürk N. Customer Dissatisfaction Towards Chatbot Services of e-Commerce Shopping Sites: A Qualitative Analysis. JTL. 2024;2024. doi:10.26650/JTL.2024.1355850.
MLA Altay, Burak Can and Naim Çetintürk. “Customer Dissatisfaction Towards Chatbot Services of E-Commerce Shopping Sites: A Qualitative Analysis”. Journal of Transportation and Logistics, vol. 2024, no. Erken Görünüm, 2024, doi:10.26650/JTL.2024.1355850.
Vancouver Altay BC, Çetintürk N. Customer Dissatisfaction Towards Chatbot Services of e-Commerce Shopping Sites: A Qualitative Analysis. JTL. 2024;2024(Erken Görünüm).


The JTL is being published twice (in April and October of) a year, as an official international peer-reviewed journal of the School of Transportation and Logistics at Istanbul University.