Large Language Models as Computational Linguistics Tools: A Comparative Analysis of ChatGPT and Google Machine Translations

Abstract

This study investigates the effectiveness of Large Language Models (LLMs), specifically ChatGPT, in machine translation and compares them with traditional tools like Google Translate. The research focused on translating speeches by King Abdullah II of Jordan, delivered in Arabic and English at significant international events in 2023. The study evaluated the translations based on meaning, functional and textual adequacy, target language mechanics, style, register, and idiomaticity. The analysis revealed that Google Translate’s Arabic-English translations were deficient, with contextual accuracy and meaning issues necessitating major revisions. The English-Arabic translations by Google Translate also required significant edits due to literal translation practices and inadequacies in several areas. Contrariwise, ChatGPT’s Arabic-English translations were rated as acceptable, needing only minor edits, and offered more natural-sounding translations. The English-Arabic translations by ChatGPT, while better than Google Translate, still showed some deficiencies but were deemed acceptable with minor adjustments. The study underscores the irreplaceability of human translators in ensuring accurate and contextually rich translations. However, the study also highlights the potential of LLMs, such as ChatGPT, to significantly enhance the translation process. Developers are encouraged to enhance LLMs’ contextual understanding and natural language processing capabilities. This involves expanding training datasets to include diverse and context-rich examples and improving the models’ ability to handle different registers, styles, and idiomatic expressions. The study strongly advocates for a collaborative model in the translation industry that integrates machine translation with human expertise to enhance efficiency while maintaining quality. These insights are crucial for driving advancements in developing and applying machine translation tools, ensuring they complement rather than replace human translators.