TranslatAR: A mobile augmented reality translator

@article{Fragoso2011TranslatARAM,
  title={TranslatAR: A mobile augmented reality translator},
  author={Victor Fragoso and Steffen Gauglitz and Shane Zamora and Jim Kleban and Matthew A. Turk},
  journal={2011 IEEE Workshop on Applications of Computer Vision (WACV)},
  year={2011},
  pages={497-502}
}
We present a mobile augmented reality (AR) translation system, using a smartphone's camera and touchscreen, that requires the user to simply tap on the word of interest once in order to produce a translation, presented as an AR overlay. The translation seamlessly replaces the original text in the live camera stream, matching background and foreground colors estimated from the source images. For this purpose, we developed an efficient algorithm for accurately detecting the location and… 
View on IEEE
engineering.ucsb.edu
96 Citations
Highly Influential Citations
6
Background Citations
37
Methods Citations
26

Figures and Tables from this paper

96 Citations

Automatic text detection for mobile augmented reality translation

A fast automatic text detection algorithm devised for a mobile augmented reality (AR) translation system on a mobile phone and a method that exploits the redundancy of the information contained in the video stream to remove false alarms is presented.

A review on using augmented reality in text translation

  • Lamma TatwanyH. Ouertani
  • Computer Science
    2017 6th International Conference on Information and Communication Technology and Accessibility (ICTA)
  • 2017
A review of literature on augmented reality text translation provides the current state of the art on research in augmented reality in text translation and permits the ways towards developing an android mobile application for real time Arabic text translation.

Implementing Statistical Machine Translation into Mobile Augmented Reality Systems

This research devised SMT methods and evaluated their performance for potential use in augmented reality (AR) systems, focusing on translation between the Polish and English languages, which presents a number of challenges due to fundamental linguistic differences.

Computer Vision for Mobile Augmented Reality

This chapter discusses the use of computer vision for mobile augmented reality and presents work on a vision-based AR application (mobile sign detection and translation), a vision -supplied AR resource (indoor localization and post estimation), and a low-level correspondence tracking and model estimation approach to increase accuracy and efficiency ofComputer vision methods in augmented reality.

CameraKeyboard: A Novel Interaction Technique for Text Entry Through Smartphone Cameras

CameraKeyboard is a text entry technique that uses smartphone cameras to extract and digitalise text from physical sources such as business cards and identification documents and is compared to a standard smartphone keyboard (Google Keyboard) and a standard (physical) desktop keyboard.

Speech enabled integrated AR-based multimodal language translation

A speech enabled augmented reality (AR) smartphones application for Hindi to English translation and vice-versa and user will be able to effectively engage, communicate and access information originally available in native language.

A Review on “ Implementation of Real-time Image and Video Processing , and AR on Mobile Devices ”

This paper reviews some improved implementations of real-time image and video processing on Face Detection and Augmented Reality fields to be used on mobile platforms and shows a remarkable improvement on Robustness and Processing Time comparing to the previous systems and exclusively Viola-Jones algorithm.

Recognition and Translation on a Mobile Device

The application presented runs on the Nokia N900 smartphone and introduces human-assisted elements into the OCR pipeline to enhance accuracy and confirms the utility of text recognition software on mobile devices.

Interactive Text Recognition and Translation on a Mobile Device

The application presented runs on the Nokia N900 smartphone and introduces human-assisted elements into the OCR pipeline to enhance accuracy and confirms the utility of text recognition software on mobile devices.

Augmented reality based mobile application for real-time Arabic language translation

An AR mobile application for real time Arabic text translation is developed that consists of three main components which are: text detection, text extraction and text translation.
...

References

SHOWING 1-10 OF 18 REFERENCES

Viewfinder Alignment

An algorithm for aligning successive viewfinder frames is described, which is robust to noise, never requires storing more than one viewfinder frame in memory, and runs at 30 frames per second on standard smartphone hardware.

Parallel Tracking and Mapping on a camera phone

An attempt to implement a keyframe-based SLAM system on a camera phone (specifically, the Apple iPhone 3G) is described and early results demonstrate a system capable of generating and augmenting small maps, albeit with reduced accuracy and robustness compared to SLAM on a PC.

Scene text extraction and translation for handheld devices

  • I. Haritaoglu
  • Computer Science
    Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001
  • 2001
An automatic sign/text language translation for foreign travelers, where people can use the system whenever they want to see text or signs in their own language where they are originally written in a foreign language in the scene.

Portable Translator Capable of Recognizing Characters on Signboard and Menu Captured by its Built-in Camera

A portable translator that recognizes and translates phrases on signboards and menus as captured by a built-in camera that offers more accurate character recognition and machine translation.

Pose tracking from natural features on mobile phones

This paper achieves interactive frame rates of up to 20 Hz for natural feature tracking from textured planar targets on current-generation phones using an approach based on heavily modified state-of-the-art feature descriptors, namely SIFT and Ferns.

Towards Automatic Sign Translation

A prototype system that can recognize Chinese signs input from a video camera which is a common gadget for a tourist, and translate them into English text or voice stream is developed.

Automatic Word Detection System for Document Image Using Mobile Devices

A real-time automatic word detection system without support of the server is proposed that shows high precision rates for one word in the central region of the image, and showed fast computational time on the mobile devices.

Translation camera on mobile phone

A mobile application, which translates Japanese texts in a scene into English by recognizing Japanese characters sensed by an onboard camera of a mobile phone and translates them into English.

Automatic text location in images and video frames

  • Anil K. JainB. Yu
  • Computer Science
    Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170)
  • 1998
Compared with some traditional text location methods, this method has the following advantages: 1) low computational cost; 2) robust to font size; and 3) high accuracy.

An edge-based text region extraction algorithm for indoor mobile robot navigation

This paper proposes an edge-based text region extraction algorithm, which is robust with respect to font sizes, styles, color/intensity, orientations, effects of illumination, reflections, shadows, perspective distortion, and the complexity of image backgrounds.