Six NMT Systems, One Language Pair: Which Best Translates Arabic-English?

Abstract

This study evaluates the quality of translations produced by six different Neural Machine Translation (NMT) systems when translating from Arabic to English. The systems under study are Google Translate, Microsoft Bing, Yandex, Systran, ChatGPT-4, and Amazon Translate. Given the precision and complexity of the Arabic language, the study aims to examine the most effective NMT system and understand how translators can utilize these tools. To achieve the study's objectives, 1,000 Arabic sentences and their English translations are examined, with established translations verified by human translators used as reference benchmarks for evaluating machine translations. Data are collected from the Tatoeba platform (2024), accessible to researchers online, and are analyzed using the Bilingual Evaluation Understudy BLEU system. The study's findings reveal significant variations in translation quality among the systems tested, highlighting the necessity for translators to be involved in the machine translation editing process. Moreover, the results indicate that ChatGPT-4 outperform other systems in producing high-quality translations. This study contributes to translation studies by offering a comprehensive comparative analysis of current NMT systems, providing practical insights for translators, and advancing research on machine translation applications.