Producer: Popular Science China
Production: Zhang Shuyu
Producer: Computer Network Information Center, Chinese Academy of Sciences
Can I still see after irreversible damage to my eyes?
Neuroscience has found that the brain has a special gift for learning to "see things" without relying on the eyes.
Image credit: veer gallery
Lying in the "massage chair", the congenital blind can also "see" the flying ball
In 1969, Nature published a science fiction paper detailing a very strange apparatus that allowed blind people to "see" things by "massaging" their backs.
Let's restore the experimental scenario:
A congenitally blind man, without any visual experience, lies in a chair similar to a dental treatment. Next to him was a vintage camera with a zoom lens.
The therapist moves the camera with a hand crank to scan the sight in front of the blind person. The resulting image is transmitted to the instrument behind him, which transmits the processed visual signal to a matrix of stimuli points on the back of the treatment chair.
Specifically, the part of the scene with weak light will vibrate the corresponding matrix stimulus point, while the part with strong light will not vibrate. These contacts directly irritate the skin on the back of a blind person, like a massage chair.
Bach-Lita invented the "visual aid artifact". The position of the lens in the picture is facing the perspective of the human eye, which can make the blind person perceive the object in front of him (Image source: Reference 2 )
After 20-40 hours of intensive training, something magical happens – blind people are not only able to distinguish between different lines such as vertical, diagonal and curved lines, but also common geometric shapes such as circles, rectangles, triangles, etc.
After learning to control the camera, through the zoom lens, the blind person can aim at different positions in the room to identify various objects such as telephones, chairs, cups (even if partially obscured) and describe the position relationship between them.
Gradually, they began to perceive the three-dimensional space in front of them:
The distance of the object, which can cause the image to change in size;
Observe the object from different perspectives, its shape will be distorted;
The backlit surface of the object casts shadows, etc. If someone throws a ball at the camera, the blind will naturally dodge.
With this "tactile-vision" instrument, blind people have even learned to recognize faces (e.g., the faces of supermodels).
Even more incredibly, they were able to "observe" changes in the appearance and mannerisms of the characters.
For example, they describe a lady who said, "She got her hair off today and she didn't wear glasses." She was moving her right hand to the back of her head. ”
The vibration pattern of the stimulus point matrix projects a two-dimensional image on the monitor oscilloscope.
The image shows the face image. After a long period of training, blind people can learn to recognize tactile patterns of similar complexity (Image: Reference 2)
Why is it that by stimulating the back, the brain can "see"?
Paul Bach-y-Rita, the principal investigator of the study, has carefully observed and studied blind people who use blind canes. When a blind person walks, he will sweep the blind cane back and forth, and the tip of the blind cane will tell the blind person the information of the road condition through the tactile receptors on the skin.
Bach-Litha was inspired: the blind cane can be seen as an "interface" between a blind man and an object. Through the pressure touch of the blind cane on the hand, spatial information such as room furnishings is formed.
Thus, the skin on the hands and their tactile receptors act like an information-gathering station. They can replace the retina, allowing images to form in the brain.
The "massage chair" is in a similar way that allows the blind to "see". Simply put, it's the brain looking at something, not the eyes.
Image source: Pixabay
The improvised brain
The results of this prospective experiment by Bach-Litha confirm the theory of "sensory substitution" (sensory substitution).
Specifically, this refers to the vital neural pathways responsible for visual function that, once broken or blocked, detour the brain.
Neural pathways that control tactile sensations were rarely used in visual perception, but now they can play as a substitute and play the role of "seeing things". It seems that the brain also understands that "all roads lead to Rome".
In fact, the brain is a lot like a decoder that stays in a dark braincase. When the various sensory information from the outside world is transmitted in, whether it is photons, air compressed waves, molecular concentrations, or pressure, texture or temperature, it will be uniformly converted into the universal language in the brain: electrochemical signals.
It is the biochemical reactions in this dark theater that form all our perceptions of reality.
Even if the perceptual signals come from extraordinary sensory neural pathways, the brain rises to the challenge of reorganizing sensory perception by constantly learning and understanding new signals.
This is thanks to millions of years of biological evolution, which has made the brain a "big coffee" with improvisation. The transformation of decay into magical super learning ability stems from the flexible plasticity of human brain nerves.
Bach-Litha was a pioneer in applying the plasticity of brain nerves to rehabilitation medicine.
After the "massage chair", some more modern designs have emerged in the world. For example, visual information is transmitted to the brain by transmitting sound to the ear, or by stimulating the forehead or tongue with small vibrations. You can "see" without using your eyes. At first glance, this sounds like a special feature. But come to think of it, this is also the result of the normal functioning of the brain.
Coincidentally, hearing can also be independent of the ear.
An alternative way of "listening" to the world
Neuroscientist David Eagleman's team has crafted a hearing aid for the hearing impaired, the Variable Extra-Sensory Transducer (VEST), commonly known as the "vest".
The "vest" comes with a microphone that can real-time perceptual coding of external sounds.
The encoded information is then mapped to some tiny vibrating motors on the "vest". Depending on the frequency of the sound, the motor activates a dynamic vibration pattern that is transmitted to the entire torso.
How the "vest" works (Image: eagleman.com )
This wearable technology can also run on mobile devices such as mobile phones and tablets. Once the device captures the sound in the environment, it maps the signal to the motor via Bluetooth.
How the "vest" works in mobile device conditions (top); the front and back of the "vest" (middle); Igman shows how the "vest" works (bottom) (Image source: Reference 3)
Adapting to the vibrating signals from the "vest" is like learning a new language. At first, these external signals were elusive. But after enough training, the brain will cross-compare different back touches, and gradually learn to extract the rules from it, converting the language of the "vest" into information that can be understood. For example, the brain can match a word to a specific pattern of vibration.
Experiments have shown that wearing a "vest" for 2 hours a day, in less than a week, the hearing impaired can spell out the words spoken by others correctly.
Eagleman said the patented technology not only saves patients from the damage of cochlear transplant surgery, but also promises to give them a direct auditory experience. It's like a blind person touching Braille to understand the meaning of words.
The story is not over, and the follow-up engineers have made another stroke! They condensed the core technology of the "vest" into a small bracelet and gave it a vivid name - "Buzz", which means "buzzing".
The bracelet's interface is equipped with a power switch, user settings button, a microphone, and a microcontroller. (Image source: Neosensory.com )
The bracelet is small, but fully functional. It is equipped with a microphone that captures ambient sound, four vibration motors, and a sophisticated signal processing system. The system can be like a "vest", the external signal through the "sound-to-touch" algorithm encoded, into the motor output of the dynamic vibration mode.
(Image source: Reference 1)
The sound signal captured by the microphone is processed by the microcontroller and converted into different vibration modes of the output of the four motors. Each vibration point is a rectangular area of 8.2mmX8.5mm
Can a bracelet with only 4 vibration stimulation points transmit enough tactile signals?
The research team monitored 18 patients who were deaf and severely hearing impaired for a month. During this period of daily life, patients wear bracelets for at least 4 hours a day. From the day the bracelet is first worn, patients undergo a round of tests every two weeks.
The data showed that patients were able to learn to distinguish between different vibration patterns converted from word audio. They can also identify vibration patterns from similar but different words.
Further studies have found that patients can also learn to recognize sounds in their daily lives. Their learning material covered 14 categories, including the sound of babies crying, car horns, alarm clocks and applause.
Happily, their recognition improves with the number of days they wear them.
It turns out that as the bracelet function continues to adapt, the wearer gradually becomes more able to perceive the world of sound waves around them.
The good story continues. I believe that the bracelet can bring hope to more hearing-impaired people.
This transformation of decay into magical learning abilities unique to the human brain opens up entirely new possibilities for limited sensory perception.
bibliography:
Perrotta, M. V. , Asgeirsdottir, T. , & Eagleman, D. M. . (2021). Deciphering sounds through patterns of vibration on the skin. Neuroscience (4).
Bach-Y-Rita, P. , Collins, C. C. , Saunders, F. A. , White, B. , & Scadden, L. . (1969). Vision substitution by tactile image projection. Nature, 221, 963–964.
Novich, S. D. , & Eagleman, D. M. . (2014). A vibrotactile sensory substitution device for the deaf and profoundly hearing impaired. 2014 IEEE Haptics Symposium (HAPTICS). IEEE.
Norman Dowich, Reshaping the Brain, Reshaping Life (2015), Mechanical Industry Press
David Eagleman, The Brain's Tale (2019) Zhejiang Education Press
Can we create new senses for humanity? | TED
https://www.ted.com/talks/david_eagleman_can_we_create_new_senses_for_humans/transcript?language=zh-cn