Augmented reality app instantly translates sign language into spoken language | Inquirer Technology

Augmented reality app instantly translates sign language into spoken language

/ 06:30 PM April 05, 2018

Image: Screengrab from YouTube/Hideaki Heng Lee

New York University (NYU) students Heng Li, Jacky Chen and Mingfei Huang created an augmented reality (AR) application that instantly translates sign language into spoken language, and vice versa.

The application, called ARSL, is one of the  selected prototypes to participate in the Verizon Connected Futures Prototyping & Development program this year.


The Connected Futures launched the initiative in partnership with the NYC Media Lab to create a platform for emerging technologies, mainly in AR, visual reality and artificial intelligence. The prototypes in the Connected Futures program showcases next-generation experiences that are enabled by 5G wireless infrastructure and an Intelligent Edge network.

ARSL uses computer vision and augmented reality which enables users to capture sign language with their smartphone camera, according to a report by Next Reality last March 29. Once sign language is captured by the camera, users will see on their phone’s screen a live translation of the sign language into their native language. ARSL also translates spoken language into sign language.


The conception of ARSL is an innovation when it comes to persons with disabilities, since trying to communicate in sign language with those who cannot speak it is already hard enough. With ARSL, the deaf and mute no longer have to feel helpless when in public, or turn to a pen and notepad to get their message across; all they need to do is just pull out their phones.

As the ARSL app is still a prototype, however, there’s no way it would be making its way to the App Store or Google Play any time soon. But this should not pose as a problem. As per a Verizon spokesperson via Next Reality, some teams, potentially including ARSL, will most likely seek commercial release in the future.

An introductory video about ARSL may be viewed below. Cody Cepeda/JB


Could a mobile app help treat mental health problems?

Sleep app, medical chat and tailored nutrition lead health tech innovations 

TOPICS: Artificial Intelligence, augmented reality, deaf, mute, New York University, Persons with Disabilities, sign language
Read Next
Don't miss out on the latest news and information.

Subscribe to INQUIRER PLUS to get access to The Philippine Daily Inquirer & other 70+ titles, share up to 5 gadgets, listen to the news, download as early as 4am & share articles on social media. Call 896 6000.

For feedback, complaints, or inquiries, contact us.

© Copyright 1997-2021 | All Rights Reserved

We use cookies to ensure you get the best experience on our website. By continuing, you are agreeing to our use of cookies. To find out more, please click this link.