Image source @ Visual China
Wen 丨 academic headlines, author 丨 Qizai, editor 丨 Li Xuewei
Have you ever experienced the world of the blind? Is it pitch black or full of mosaics?
What we see every day is accustomed to: gray concrete forests during the day, colorful neon at night, and if we lose these habits, what will happen to life?
Tencent Tianmei team once developed such a game to restore the world in the eyes of the blind - "See", in addition to the necessary operation of the roulette wheel and the white outline that symbolizes the touch, "See" is almost all composed of black, players need to play a blind person, use the blind cane and grope way to feel the outline of the object, find a way to travel, experience the blind person's life for a day.
Although the development team adjusted the difficulty of the level appropriately, it was still impossible to complete the game mission easily in the game with an almost blind world.
However, in fact, the life of a blind person is more difficult than the experience in the game. It's hard to be completely empathetic to their difficulties, but it's imperative to help them as much as possible.
Recently, in order to help visually impaired people live better, a group of high school students created such an iPhone application for visually impaired people - Athiea.
Figure | Team members (Source: atheia.org)
Athiea can observe the user's environment and increase the user's awareness of space and safety. Its facial recognition features recognize family and friends, and a built-in text reader can scan words and read them aloud. The scanning function identifies, counts and precisely locates objects in the user's field of view.
It can also track its own observations. Users can ask for the location of the desired item, and Atheia responds to the user's questions with its location, the time it was last identified, and nearby objects.
In addition, its search function guides the user to objects in the environment by giving them haptic feedback and audio commands.
Figure | Procedure (Source: IEEE Spectrum)
Inspiration: Computer Vision
"We noticed that the field of computer vision has reached a point where objects can be detected very accurately and described like a human," Choi said. "Inspired by this technology, we have improved the accessibility of the technology and made it better applicable to the real-world scenario of providing visual assistance to visually impaired people."
Atheia can answer many real-world questions, such as the shape and size of an object or the time of day.
When they feel they are in a dangerous environment, users can activate the app's sentinel mode by voice commands or by pressing a button. Atheia will start recording videos, send real-time updates to emergency contacts, and provide contacts with the user's location.
The young people began their research on the project in 2019. Before getting a working prototype, they tried many different product forms.
The app is now being tested by volunteers from Maryland's Blind Industries and Services, a Baltimore-based nonprofit that provides training and career resources for the visually impaired in the state.
Mentoring this group of talented young people is IEEE member Pamela Ahn, director of the school's Electronics Research Laboratory and founder and president of the IEEE Richmond (Va.) Section Women in Engineering affinity group.
"These four kids want to make a difference in the world," Ahn said. "They want to help others, and that's from a very selfless standpoint. They are smart, hardworking, and focused. ”
The construction of the technology
This APP uses information processing channels, search and obstacle avoidance algorithms to achieve the purpose of performing user requests with low latency and high precision. The app uses the latest computer vision models and machine learning techniques to improve the accuracy with which it classifies objects.
It also includes a leading object detection model, a multimodal framework for visual and linguistic research, a visual question-answering program and a text recognition algorithm.
Data is processed locally on mobile devices, rather than through cloud services, so user requests can be fulfilled regardless of whether there is a cellular signal.
"Through testing, we realized that no matter where the user is, the service is the most important," Ravella said. "Converting our large and heavy computer vision model into one that can run on the iPhone was a huge challenge. However, thanks to the iPhone's neural engine, this is all possible. "
Ravella says the app is currently only available on iOS because the iPhone is the most popular among people who tested the device, and the software has more accessibility features. It also allows students to shift hardware development to Apple, he said, and focus more on software, which has a great iPhone camera, depth technology, processing power, and battery life.
Blind assistive devices on the market can be as high as $6,000, but Atheia users only need to spend about $10 a month on a subscription.
Value user feedback
The students wore gloves with cameras when they first tried to use the assistive device, but the experiment was not successful because it was designed without potential user input.
"We fell into a trap that a lot of people would encounter," Choi says, "and we first created a solution and then we found the exact problem that matched, but these problems are not the problems that you should solve with these devices." Due to the limitations of the epidemic, they could not conduct face-to-face interviews, so the children posted on Facebook to find visually impaired volunteers to consult with them about the top needs of the visually impaired.
Figure | Nafi (left) and Choi (right) are gluing together parts of a 3D printed prototype bracelet that will display the Atheia app. This is one of six prototypes tested by the team but rejected (Source: IEEE Spectrum)
"They're willing to give us feedback, they inspire us to keep innovating, and they're looking forward to our research," Ravella said.
Based on this feedback, the team abandoned gloves and began developing wrist-worn and head-mounted devices, and eventually developed mobile apps. They made sure to get input about each of their six prototypes.
To understand which technologies to use, Ahn helps students obtain research articles from a variety of IEEE publications, including IEEE Spectrum.
After spending hundreds of dollars on parts and paying for software, the kids found a partner to help cover the cost of development. Their partners include Amazon Web Services, the Maximus Foundation, MIT's Assistive Technology Division, and Ultralytics.
Young developers envision the future
When it comes to future career fantasies, the four students invariably said that they all intend to pursue STEM careers.
Ravella says he wants to study the application of machine learning in cybersecurity. During his internships at MITRE and Columbia University, he learned that hackers were too easy to break into IoT devices, which was a cause for concern.
Nafi wanted to be involved in computer vision research, which incorporated "hybrids of physics." He has worked on digital pathology at Dartmouth College.
Choi still wants to pursue a career in computer vision. He really enjoyed the Atheia project and another project he developed using computer vision during his internship at George Mason University.
Khondaker said he wants to continue to work in the field of artificial intelligence. He says he enjoys developing the customer-centric aspect of Atheia because "you're meeting the people you're designing projects for."
At present, the APP has its own homepage, which introduces the promotion process and application technology of the project in detail.
Only when you truly sink into the darkness and feel that you are derailed from society will you understand the loneliness of blind friends.
At present, there are still many problems in the market A app with visual impairment function, and urban blind lane planning is not taken seriously. The emergence of this APP has undoubtedly brought the gospel to the visually impaired. Even though it still looks a little childish, it is still a good product full of social responsibility.
I believe that in the future, after these four teenagers have achieved success, they will bring this visually impaired person's "good helper" to those who need them, and help thousands of visually impaired people live better.
Reference Links:
https://spectrum.ieee.org/iphone-app-for-visually-impaired
https://www.atheia.org/