16+

Assessing promise and limitations of ChatGPT: analysis of translation quality and translation errors of large language models

The rapid advancement of natural language processing (NLP) technologies and the increasing integration of automated tools into translation workflows necessitate a comprehensive exploration of the capabilities and constraints of large language models (LLMs), such as ChatGPT, across various stylistic domains and thematic areas. This study addresses the need for a comparative analysis of machine translation quality produced by ChatGPT, specifically focusing on specialized, literary, and scientific discourses. Central to this research is the identification of errors associated with contextual misinterpretation and the phenomenon of model "hallucinations."

Based on a comparative framework, the research delineates the operational parameters of ChatGPT, including its potential for context recognition, communicative situational awareness, and the resolution of specific translation challenges at the levels of stylistic congruence and lexical equivalence. This research employs a hybrid evaluation framework that integrates a linguistically-grounded metric with the automated BLEU scoring system. The findings reveal a significant correlation between translation quality, prompt precision, and the typological characteristics of the source text. Both linguistic and automated metrics indicate that translations of highly specialized technical content exhibit higher accuracy than those of literary or scientific texts, which require nuanced syntactic construction and terminological selection within specific linguistic conventions. Furthermore, the results highlight inherent risks of generative AI, such as semantic distortions and contextual errors, which can compromise the integrity of translated or edited content. The paper concludes by proposing practical strategies and prompt engineering techniques to enable translators to effectively leverage innovative neural network technologies, particularly the latest iterations of ChatGPT, in professional practice.

Number of views: 9 (view statistics)
Количество скачиваний: 8
Full text (HTML)Скачать XMLTo articles list
  • User comments
  • Reference lists
  • Thanks

While nobody left any comments to this publication.
You can be first.

Leave comment: