AR headset names objects to help blind people | Inquirer Technology

AR headset names objects to guide blind people in new places

/ 06:44 PM December 03, 2018

To assist the blind in navigating unfamiliar places, scientists at Caltech in the U.S. state of California have developed an augmented reality technology housed within a portable headset that gives objects a voice.


This week, Caltech announced the publication of a paper (“Augmented Reality Powers a Cognitive Assistant for the Blind”) outlining the work of scientists who “have combined augmented reality hardware and computer vision algorithms to develop software that enables objects to ‘talk.'” In layman’s terms, these researchers have constructed a headset powered by AR technology, called CARA, that says the name of objects in particular pitches and in particular locations to inform the wearer of what and where their surroundings are.

augmented reality

CARA, an AR assistant developed to help the blind navigate new places, was built for Microsoft’s HoloLens. Image: Microsoft

CARA stands for Cognitive Augmented Reality Assistant, which is the tech that is used within Microsoft’s HoloLens, a wearable headset computer that recognizes objects. The tech uses “spatialized sound,” or localized voices, to alert the user of where a particular object is — the closer an object is, the higher pitched “its” voice will be.


Of course, if the scientists stopped there, the headset will be saying tons of things to the wearer simultaneously which could no doubt be pretty alarming. Instead, the team programmed CARA with three modes to make understanding a space less overwhelming. Spotlight mode programs CARA to voice object names only where the user’s gaze is targeted at them. The second mode, scan mode, lists all objects in the environment that the headset can identify from left to right. Lastly, target mode allows the wearer to select a single object to “speak exclusively” with to use as a guide.

Although in its early stages, the technology is promising as all visually impaired individuals who tested the headset were able to successfully navigate a route guided by CARA on the first try. With algorithms for computer vision evolving and developing at such a rapid pace, the team looks forward to the possibility of this AR headset being used at banks, hotels, and malls, basically anywhere that navigating could be a challenge for the blind, sooner rather than later. NVG


Sharp vision: New glasses help the legally blind see

How the blind use clicks to ‘see’ – study

The latest tech news delivered to your inbox

By providing an email address. I agree to the Terms of Use and acknowledge that I have read the Privacy Policy.
TOPICS: AR headset, augmented reality, blind, Caltech, disability, navigation
Read Next
Don't miss out on the latest news and information.

Subscribe to INQUIRER PLUS to get access to The Philippine Daily Inquirer & other 70+ titles, share up to 5 gadgets, listen to the news, download as early as 4am & share articles on social media. Call 896 6000.

For feedback, complaints, or inquiries, contact us.

Subscribe to our technology news

By providing an email address. I agree to the Terms of Use and
acknowledge that I have read the Privacy Policy.

© Copyright 1997-2022 | All Rights Reserved

We use cookies to ensure you get the best experience on our website. By continuing, you are agreeing to our use of cookies. To find out more, please click this link.