Augmented reality app instantly translates sign language into spoken language

Image: Screengrab from YouTube/Hideaki Heng Lee

New York University (NYU) students Heng Li, Jacky Chen and Mingfei Huang created an augmented reality (AR) application that instantly translates sign language into spoken language, and vice versa.

The application, called ARSL, is one of the  selected prototypes to participate in the Verizon Connected Futures Prototyping & Development program this year.

The Connected Futures launched the initiative in partnership with the NYC Media Lab to create a platform for emerging technologies, mainly in AR, visual reality and artificial intelligence. The prototypes in the Connected Futures program showcases next-generation experiences that are enabled by 5G wireless infrastructure and an Intelligent Edge network.

ARSL uses computer vision and augmented reality which enables users to capture sign language with their smartphone camera, according to a report by Next Reality last March 29. Once sign language is captured by the camera, users will see on their phone’s screen a live translation of the sign language into their native language. ARSL also translates spoken language into sign language.

The conception of ARSL is an innovation when it comes to persons with disabilities, since trying to communicate in sign language with those who cannot speak it is already hard enough. With ARSL, the deaf and mute no longer have to feel helpless when in public, or turn to a pen and notepad to get their message across; all they need to do is just pull out their phones.

As the ARSL app is still a prototype, however, there’s no way it would be making its way to the App Store or Google Play any time soon. But this should not pose as a problem. As per a Verizon spokesperson via Next Reality, some teams, potentially including ARSL, will most likely seek commercial release in the future.

An introductory video about ARSL may be viewed below. Cody Cepeda/JB

RELATED STORIES: 

Could a mobile app help treat mental health problems?

Sleep app, medical chat and tailored nutrition lead health tech innovations