Recent DARPA-sponsored MT evaluations showed that phrase-based statistical MT systems derived through automatic training on very large corpora achieve the highest translation quality for both generalized and specialized texts. Intuitively, syntax, morphology, and semantics should help; however, rule-based systems that incorporate such knowledge are not yet able to beat the best statistical systems. We propose the following plan for incorporating richer linguistic knowledge into the translation process: Start from a good baseline: the best A2E machine translation system in the world (as evaluated by NIST). Devise experiments to determine major sources of error in the system (E.g.: verb agreement; morphological errors; contextual errors) Assess the impact that solving these problems may have on translation quality. This is done by measuring the increase in BLEU score when a problem is fixed. Assess how our current system should be enriched to deal with each of these problems. Rank-list the problems in order of impact on translation quality and feasibility of addressing them using richer linguistic knowledge (morphology, syntax, semantics, context). Devise solutions for solving these problems in turn and incorporate the solutions into an end-to-end system
Benefits: This project seeks to develop an understanding of the distribution of translation problems observed in an existing phrase-based statistical machine translation system (Language Weavers Arabic->English phrase-based statistical translator, currently in alpha testing in-house and scheduled for a first release in October 2003), and to prioritize the task of addressing the problems based on experiments designed to reveal which problems have the most impact on translation quality. A follow-on Phase II project would implement changes to address the highest priority translation problems, and incorporate them into a deliverable translation system. While machine translation software has been commercially available for many years, it has never been a large scale commercial success. This limitation is primarily due to the low baseline quality achieved by current translation technology, and the difficulty, time, and expense of customizing machine translation for specialized subject areas and text types. Language Weaver has already made significant strides with respect to both of these limitations, and is in an ideal position to conduct and productize the proposed research. The present proposal focuses on raising baseline quality. Raising baseline translation quality: Language Weaver was formed to commercialize research in statistical MT conducted at the University of Southern California, Information Sciences Institute. Arabic->English and Chinese->English translation systems developed by the research team at USC/ISI recently outperformed all other commercial and research systems in formal evaluations by DARPA, involving both human and automated evaluations. Language Weaver is currently productizing the Arabic->English translation system under contract from In-Q-Tel, a venture capital company formed by the CIA to foster commercialization of technologies anticipated to be valuable to the intelligence community. Language Weavers Arabic->English translation system offers the highest known translation quality as a starting point for the present research project. Cost-effective customization: Historically, high quality automated translation has only been achieved when a machine translation system is thoroughly customized to the subject area and text type on which it will be used. With rule-based systems, which represent the state of the practice in commercial machine translation, the time and labor required to perform this customization generally eclipse any cost savings anticipated from automating the translation process. Language Weavers statistical learning technology offers rapid, fully automated customization. This topic is not addressed in this proposal, but is explored somewhat in the related work section. Productization and commercial deployment: Language Weaver is a relatively new company, founded in early 2002. Our first two US Government funded language pair translators (Hindi->English, Somali->English) are currently in beta release, with production release scheduled for September 2003. This will be followed by a production release of Arabic->English in October 2003. We will launch our first commercial products in September 2003 as well (English->Chinese, French, Italian, Spanish), and are already conducting beta tests with a number of major multinational corporations. The core issue in the present project is raising the quality ceiling that machine translation can achieve for Arabic->English translation. Language Weaver has a highly qualified staff of developers within the company, as well as the brain trust at USC/ISI to draw on. Our Arabic->English translation system offers the highest possible starting point, together with an open, flexible architecture that allows hybridization and extension. Finally, Language Weaver offers a strong channel for productization and commercialization of research conducted in the context of this project.
Keywords: statistical machine translation, phrase-based translation model, automated translation evaluation, translation quality
