SBIR-STTR Award

Most DoD missions occur in areas of the world where the native language is not English. The warfighters find themselves in situations where a quick, real-time translation of signs, placards, and documents could lead them out of dangerous situations or provide important clues and information relative to the mission at hand. In places such as Afghanistan or North Korea, this turns out to be a particularly difficult problem, since the languages commonly used in these countries are written with characters that cannot be typed into a translation device or even searched for in a dictionary (unless the user has a significant knowledge of the given language.) A valuable tool would be a handheld device (e.g., a mobile telephone or PDA) that would be capable of translating the text found in natural scenes into spoken English and text. The device should also possess the ability to transmit the captured image and text information to other warfighters in the area or to the warfighter’s command and control organization or intelligence agencies that may assist in understanding the image as it applies to the larger overall mission. The Rosetta Phone takes the form of a handheld mobile device (HMD) such as a mobile telephone or a PDA. The HMD is assumed to be equipped with a color camera and, at least part of the time, wireless network capabilities. In order to be able to provide a translation of text found in natural images, the HMD needs to be programmed to perform the following sequence of tasks

Keywords:
Translation, Cell Phone, Wireless, Optical Character Recognition, Pda

Unable to read or interpret written text can pose major problems for people in many circumstances, and Rosetta Phone is a system that will aid in interpreting this written text. The objective of this research project is to develop a complete system investigating both hardware platforms and software programming environments capable of real-time, autonomous, visual interpretation of written text. In Phase I, the proof-of-concept system developed shows that the Rosetta Phone concept is achievable using Handheld Mobile Devices (HMDs) available on the market today. Using open standards we have developed a system which will run not only on HMDs, but also on any computer running Linux or other UNIX-like operating systems, without making any modifications to the code. All the tools developed in Phase I are able to run on the device, and are able to process an image and generate the desired output. Phase II will focus on improving the software, expanding the database of characters and languages, and continuing to research commercially-available and militarized HMDs to ensure sufficient camera resolution, data storage, memory, and processing capabilities. Improvements to the software will include image fidelity, processing, and additional language support. Rosetta Phone could have military applications in foreign countries where translation of signs or other text could provide vital information to the warfighter. Other obvious applications are a reading assistant for the blind or as a portable translation device for tourists to foreign countries.

Keywords:
Segmentation Optical Character Recognition (Ocr) Text Translation Handheld Mobile Devices Foreign La