TO THE QUESTION ABOUT THE MAIN CRITERIA FOR EVALUATING MACHINE TRANSLATION

Abstract

Assessing the quality of machine translation is a complex task that requires attention to detail and a comprehensive approach. The use of different evaluation metrics and their combination allows you to get a more accurate and objective view of the quality of automatic translation. A practical approach to quality assessment helps to improve results and provide users with the best possible experience. This article examines the aspect of normativity of the criteria used in evaluating translations produced by machine translation systems (MT). The problem raised in the article is an important stage in the systematisation of approaches to evaluating MT, on the one hand, by a human in the context of the methods and criteria used by him/her, and on the other hand, by an automated system and its metrics. The article emphasises the need to distinguish between the criteria for evaluating machine translation by an automated system and traditional human translation. Particular attention is paid to the study of translation evaluation methods, such as the method of reverse translation, the method of evaluating the quality of translation by determining the degree of receptivity for the consumer, etc. It is emphasised that the requirements for assessing the quality of traditional translation and translation performed by MT systems are not identical in terms of norms. The article contains a study of automatic evaluation metrics. The author emphasises the need to choose the appropriate metric for this purpose, or to combine elements of several metrics, since each of them has both advantages and disadvantages, and the choice of the right metric can significantly affect the evaluation results. For example, the BLEU metric may be useful for general evaluation of translation accuracy, but it does not take into account word order and context, which can lead to miscalculations in determining the quality of a translation. METEOR, on the other hand, takes word order into account, but may be less sensitive to certain types of errors. TER allows you to determine the number of errors in the translated text and determine its quality, but is less sensitive in terms of overall translation accuracy, as is WER, which helps determine the accuracy of automatic translation and the number of errors in it, but does not take into account word order or certain types of errors; and CER helps assess translation accuracy only at the level of individual characters.