3 Metrics for Measuring Language Translation Quality

April 07, 2011

All members of a translation team face interesting challenges. For instance, an Arabic linguist or reviewer working on an Arabic translation project can run into well meaning "interference" from a client that actually decreases translation quality. GPI's Arabic Globalization Specialist, Abduljalil Al-Juboory, shares an interesting experience with "good intentions gone bad" that led to some thoughtful insights for judging translation quality.

Read further into this blog to see the careful balance that must be kept between client guidelines and expectations weighed against professional linguistic knowledge about the target language. Although the best practices in this blog will apply to many languages, they are especially applicable to Arabic, which is a very rich and highly expressive language.

The customer is always right

I was not surprised when our account manager told me that our client wanted alternative text in an Arabic translation project. Arabic is a highly expressive language and clients sometimes receive internal feedback from in-country staff based on particular preferences. "Clients are always right," the account manager explained. After all, it was only one headline for a marketing brochure.

So, I submitted two more options for the client to choose from. Unfortunately, the client's General Manager rejected what I knew to be good Arabic translation. So I had to ask our account manager, "How can a non Arabic-speaking General Manager judge or measure Arabic translation quality by himself?" I was lucky it was only a single headline!

The answer came as quite a shock: The GM simply said that the English headline consisted of 8 words, while the Arabic version only had 5 words; he had heard of "text expansion" and "knew" that Arabic should always be longer than the English. So, the GM copied and pasted the English headline into an automatic translation website tool until he got a result of 8 words translated into an Arabic headline. Then he sent it back to us to use it as a headline for his strategic marketing campaign! The text he submitted did not come anywhere near conveying the message of the original English source text.

Recognizing the need for professional help in language translation

Is this a professional or an effective way to evaluate translation quality? I honestly don't blame the General Manager, but I definitely put the blame on members of his staff, who convinced him that an Arabic sentence must always be longer than the English source.

I am not discussing the features of every language, and I am not judging how creatively each language can be used. My main concern is the quality of translation and how it can be effectively measured. To achieve this result, I recommend some metrics that can really help clients, proofreaders and reviewers when measuring translation quality and accuracy. To keep the process simple, I will only focus on three major points:

  1. Translation Accuracy and Product's features
  2. Translation Structure and Grammar
  3. Language and Style (Creativity) in Translation

(1) Translation Accuracy and Product's features

There is no room in this area for extra creativity or style preferences; the intent of the source author is either accurately conveyed through translation or not. Ultimately the final judge is the product's owner, the client himself. The translator's role is only to render the product's features into another language; any failure to do so may affect the final decision to accept or reject the entire project. In this instance, I would say yes, the client is usually right.

However, the translator should have a right too to suggest alterative ways to convey the message in a more effective manner, taking into consideration many factors such as cultural sensitivity and the precise locale of the targeted audience, if he is translating for his own local market.

(2) The role of Structure and Grammar

Here, I am discussing accuracy in structure vs. accuracy in meaning. No translation work is acceptable unless it uses the right structure and correct grammar; these components are the brick and mortar for any kind of written work. In order to convey an accurate message in the target language, we should align those bricks in the right direction, using the proper kind of mortar to stick them together. In this instance, there is little room for creativity, as no creative style is acceptable when accompanied by broken grammar. This is especially true when working with highly sensitive languages like Arabic.

A linguist or translator cannot use an adjective in place of an adverb, nor shift a prepositional phrase into a verbal clause, and then claim that "he went creative." Each "figure of speech" has its own purpose, which should be utilized correctly within the sentence. Combined, all components should deliver clear, easy-to-read text and a message that will be positively received by its target audience. This is the translator's role, but the client's role is not any easier.

The "Product's Owner" or client is on the other side of this process, and his role is to make sure he received the translation work he wants and the quality he is looking for. When it comes to judging translated text, at a minimum the client (or his staff) should possess at least basic knowledge of the target language and grammar; this will save a huge amount of time and effort for both parties. The only way that clients can ensure having accurate translation for an expressive marketing message in languages like Arabic is to have linguistically competent staff review their translation. Naturally, contracting with a reputable and accredited translation company or language service provider is a key ingredient to success in this endeavor.

(3) Language and Style (Creativity) in Translation

Let's go back to my first example about the General Manager, and why he believed Arabic language text should be longer than English. Through my experience as an Arabic reviewer, Arabic proof-reader and an Arabic evaluator, I have noticed that many Arabic translators use longer sentences than necessary, especially when they work on marketing or advertising translation projects. In order to show off their creativity and linguistic skills, some Arabic linguists do everything possible to "enrich," and sometimes even overload the text, by using traditional, classic and even "heavy weight" words and terms.

Many of these "fancy" terms are no longer used in modern Arabic language for marketing purposes. Arabic linguists like this are killing their text by "over doing" it. When we ask them, "why do you use so many of these extra terms, which are not adding any value to translation quality," linguists in this category answer: "we went creative!" Does creativity really mean long, overdone, yet "boring" text? I don't think so.

Accommodating cultural and accuracy issues

Arabic language and culture are both very rich. In spite of this, Arabic people have a saying: "خير الكلام ما قلّ ودلّ" --   which has an equivalent in English that means "Brevity is the soul of wit." This traditional statement clearly ends our argument. Keep your text simple and easy. Shorten it as much as you can, without compromising the original message and the beauty of the style. For me, real creative writing or creative Arabic translation is to combine or convey a huge amount of meaning in just a few words.

Another point related to this topic is the issue of following the source language author's style. Some Arabic linguists feel negatively about this, thinking that close adherence to source language style inhibits their creativity. I would say "why not?" If the source text's style is great, why should we change it? Be aware that unless instructed otherwise, translators have the freedom either to adopt the source or target language point of view, without being accused of being too literal or to stray far from the original text in an attempt to be creative.

In summary, let's all revisit our methods and strategies to review and judge the quality of any translation work.

Further Resources on Arabic language, Arabic Translation and Arabic Localization

Globalization Partners International (GPI) has extensive experience with a variety of clients marketing to the MENA region, localizing marketing materials, technical documents, and large, scalable websites into the Arabic language. Due to increased demand for projects in Arabic and other BiDi languages, like Farsi, for projects aimed at consumers in the MENA region, GPI expanded its facility in Dubai in 2010: " GPI Translation Office in Dubai (UAE)".

We have previously posted a number of useful guides on best practices in this area of translation. Three of our previous blogs may be particularly relevant if you are interested in reaching Arabic-speaking consumers in a more effective manner:

Globalization Partners International has created comprehensive information on website globalization for Arabic-speaking consumers in a white paper,  Website Globalization and E-Business United Arab Emirates, which is available via a free download. Feel free to contact GPI at info@globalizationpartners.com or at 866-272-5874 with your specific questions about this market and your project goals. You may also request a Translation Quote for your project as well.

Category:
Document Translation
Tags:
Translation, localization, project management, Arabic, translation quality, text expansion

6 Considerations When Preparing for Software TranslationHow to comply with new Chinese “Wealthy Lifestyle” advertising guidelines

Comments

  • benjamin wegenerOn Aug 03, benjamin wegener said:
    Reading the word "metrics" in the header, I had become very curious in what way quality in translation can be measures.
    Since I work in Film Localization, I have often thought about how to develop metric to assess a dub in order to support dubbing decisions in conversations with Finance and clients.

    Although the points made in this post are well made and essential guidelines for translation and adaptation, they do not offer a measurable approach to translation quality - they are still vague as they rely on subjective assessments (except maybe in terms of grammar, which, though, is also to a certain degree open to preferences).

    So, the holy grail of metrics for translation has still not been found.
  • Abduljalil Al-JubooryOn Aug 04, Abduljalil Al-Juboory said:
    Thanks Benjamin! For sure, measuring quality, and each professional's view of what constitutes quality, can vary. We have worked under very interesting guidelines for QA with a wide variety of clients. I hope the blog gave you an idea of some of the challenges with QA and the importance of trying to measure it in order to improve it.
Contact Us FREE Globalization eBooks Request Demo Request Quote


Federico has over 12 years' experience as a globalization engineer managing a wide range of software and website globalization projects (internationalization I18n + localization L10n). His expertise spans software and website internationalization and localization processes, standards and tools as well as locale specific SEO. Federico has completed hundreds of successful globalization engagements serving as lead I18n architect involving different programming languages. He is a certified developer in several content management systems and helps clients create world-ready applications, utilizing development practices that are faster, more economical, and more localization-friendly.