AI Labs
Large Language Models for Machine Translation Quality Annotation: Humans and Models Are Both Challenged
Large Language Models (LLMs) are considered to be a more efficient and cost-effective alternative to human judgment for Machine Translation (MT) evaluation. With MT evaluation spanning a large number of language pairs, domains and levels of annotation granularity, LLMs must be thoroughly evaluated across these dimensions before being reliably used as alternatives to human evaluation. In this paper, we evaluate the performance of LLMs for two prominent MT quality evaluation schemes: Multidimensional Quality Metrics (MQM) and Error Span Annotation (ESA) by comparing their agreement with human…
You're reading a preview. The full article is published by Apple Machine Learning Research on their website.
Read the full story on Apple Machine Learning Research
