In 2022, there are only two games in the entire gaming circle, Elden Ring and the other.
Throughout the year, the action RPG game nicknamed "Old Man Ring" sold more than 20 million copies worldwide, won numerous game awards, and became a masterpiece in the hearts of game fans around the world.
It is not unusual to clear all kinds of fancy levels, but if someone does not use their hands, only rely on "thinking", and also play the "old man ring", you will be somewhat curious.
Game streamer Perri Karyal made such an attempt.
Earlier this year, she showed her first on Twitch that she used brainwaves to manipulate her character in Elden Law Ring, defeating one of the game's hardest bosses, "Valkyrie" Marlenia, when some of her movements also required the help of controllers. By May, she had upgraded her device and successfully demonstrated for the first time that she could play Elden Law Ring on Xbox entirely through brainwave peripherals without any controller assistance.
How did she do it? Can ordinary people replicate her operations? How far are we from brain-controlled games in science fiction? Could this be a new opportunity for gaming startups?
01 Play games with "brains"
How can I manipulate the game with brain waves?
Unlike ordinary people's imagination, Perri Karyal does not use brain wave manipulation to fight monsters by imagining actions such as "running forward", "dodging", and "attacking" in real time in his brain. She manipulates the game character by imagining specific actions that are better recognized by the EEG device she wears, and then binding these actions to specific buttons in the game.
"When I'm about to attack, I imagine pushing something very, very heavy away from my body, and when I heal, I imagine that I pull and let something fall on my face." Perri Karyal describes.
When the human brain performs different behaviors, brain waves show different patterns, and these patterns are detected in real time by the EEG device she wears. Perri Karyal uses images in a Youtube video to visually illustrate the difference between different brain activities.
Perri Karyal's brain reaction as he speaks | YouTube
Perri Karyal's brain reacts when imagining an "attack" | YouTube
The first image captures the moment she speaks, and it can be observed that multiple brain areas of her brain are activated while speaking. And as she struggled to push a weight away from her, her brainwaves showed a unique pattern that was clearly visible on the second image. Once the device she was wearing successfully recognized the pattern, the EEG device sent instructions to perform her pre-programmed attack action.
Perri Karyal has a master's degree in psychology and has used electroencephalogram (EEG) as a research tool extensively in her academic research. As a simple, painless means of recording the electrical activity of the brain, EEG can observe the work of different areas of the brain. In addition to researchers using it to provide information about brain activity, doctors also use electroencephalograms to test for diseases such as epilepsy. Medical EEG equipment should have been seen in movies, like the actor's bald hood, which is covered with many electrodes.
"By chance, I came across an EEG device with Bluetooth, and my device budget just covered its cost, and I thought, I must try it." Perri Karyal mentioned. In this way, she began to contact non-medical-grade EEG devices, and thus began her journey to use these devices to control the game, which was out of control.
"I'm using equipment that was meant to be contextual research," Perri Karyal says, which refers to the study of insights gained by observing people's activities in everyday situations. "Such EEG devices cannot be diagnosed or used in medical scenarios."
According to Vice reports, Perri Karyal was supposed to use EMOTIV's device, EPOC X. Because it is not a medical-grade EEG device, its output is relatively limited, and it can only recognize the four actions of pushing, pulling, picking up and putting down. After recognizing these operations, the device itself provides some built-in operations, as well as APIs for people to access other devices for control.
For Perri Karyal, these four actions are enough for her combat and healing operations in a game like Elden Law Ring, but other actions require some controller assistance. So she finally chose to install a gyroscope on the brain-controlled device, and let the sensors cooperate with each other through APIs and some python programming to help her walk freely in the game and complete other operations. She is also trying to add richer features such as eye tracking and expression recognition to complete richer interaction in game scenes.
02 "Brain-machine" is not easy
Seeing this, you may be thinking, it sounds like Perri Karyal's operation is not very difficult, such a cool way, why don't the games on the market add the option of "mind manipulation" to play?
In fact, although ordinary people can indeed achieve brain-controlled games in a similar way, and some companies have used EEG devices to make brain-controlled games, game companies still face many obstacles if they want to launch brain-controlled games, use EEG as the main or even the only interaction method to achieve a good complex game experience, and be widely commercialized.
One of the problems is the wearing comfort of EEG devices.
EEG devices record the brain's electrical signals through electrodes attached to the scalp. Electrical signals originating in the brain need to pass through the skull and scalp to be picked up by EEG devices, which means that there is a lot of noise and error interference during the measurement. Usually we measure EEG in hospitals, and in order to ensure accuracy, conductive paste is applied to the electrodes in contact with the scalp to make the current transfer effective, fast and uniform.
But the conductive paste itself can stain the hair. It is extremely inconvenient to wash your hair after every time you wear an EEG device. Moreover, expertise is required for the preparation of conductive pastes, skin cleansing, or to avoid short circuits of adjacent electrodes caused by gel diffusion during testing.
As a non-medical device, the Emotiv EEG device used by Perri Karyal has taken electrodes that are already relatively comfortable, requiring only normal saline to make the electrodes more conductive. But even so, in order to keep the device effective during the live broadcast, she still needs to constantly wet the electrodes with normal saline.
Perri Karyal needed to constantly wet EEG's electrodes with saline|YouTube
As a geek player, the above options may be acceptable, and if you want to become a large-scale commercial consumer product, the comfort of EEG devices is obviously lacking.
Another problem is a common problem that exists in all brain-computer interface solutions. Although the electrical signals of each person's brain have certain commonalities, they are ultimately different. In any application of brain-computer interface, there is a process of personalized adaptation. And this adaptation process, the time often varies from person to person.
In fact, although the Emotiv device used by Perri Karyal can only recognize four actions, it took Perri Karyal a long time to personalize and calibrate the four movements in order to make them accurate enough. "I spend more time training EEG than I do in games. I spent over 250 hours on the game." Perri Karyal has spoken. Such a lot of training time can be described as very amazing, and the average user may be unable to bear it.
In addition, the cost performance of brain control devices may also be a large-scale commercialization problem. The accuracy of the equipment used by Perri Karyal is lower than that of medical grade, but its price is not low - $849. The price of the device corresponds to the accuracy of the input, the computing power of the chip, and the ease of operation of the platform. Even so, the output operates in only four gears. She needs to install a gyroscope and other things herself to complete such an amazing fully brain-controlled operation in a game like "Elden Law Ring", which requires a high timeliness of fighting, but the fighting method is not particularly complicated.
Although with the development of hardware, the price of brain control equipment ranges from several hundred yuan to several thousand yuan, and there are many choices of gears - but the output of lower price brain control equipment will be relatively limited. Players can play simple brain-controlled games, or use brain-controlled as an aid to complex games, but it is difficult to achieve complete brain-controlled operations in complex games.
03 Commercialization direction of brain-computer
Brain-computer interface technology that uses signals to manipulate machines by acquiring EEG signals can be roughly divided into two categories: invasive and non-invasive. The former refers to the invasive implantation of electrodes into the human brain to detect EEG signals, and then use the signals to do more. Neuralink, a brain-computer interface company co-founded by Musk and recently approved by the FDA for human experiments, falls into this category. And EEG technology is representative of non-invasive.
Invasive brain-computer interface technology, which does not need to penetrate the skull and scalp to obtain EEG signals, is naturally clearer than non-invasive methods to obtain electrical signals. The biggest advantage of non-invasive brain-computer interfaces is that there is no risk of surgery. Gaming, as an entertainment application scenario with fault tolerance, seems to be naturally more closely related to non-invasive brain-computer interface technology.
Therefore, although there are still many obstacles to the immediate large-scale commercialization of EEG brain games, it does not prevent the use of EEG brain games is still an extremely attractive application scenario. Many entrepreneurs are also trying to circumvent these obstacles and find the right niche for it.
At present, a more mature direction is the game of attention monitoring direction.
Since the neurons responsible for attention in the human brain are mainly distributed in the forehead area, if only attention needs to be monitored, the electrodes can only be attached to the smoother forehead. Monitoring the forehead is relatively less difficult than in areas with dense hair. Monitoring can be performed using only a headband device, without putting excessive pressure on the head or using saline frequently to moisten the hair, and the overall experience is much improved.
Although compared to Perri Karyal's controls, attention games are a bit monotonous: the game mode is often focused so you can advance or defeat enemies. But as a purely novel experience, it still holds. Some companies will choose to deploy such games in shopping malls for the public to pay for, or develop toys such as "mind" controlled drones for sale.
In addition to being purely entertaining, such gaming experiences actually have real medical value.
Domestic brain-computer interface startup Qiangbrain Technology has a similar product: for attention-deficit children, first through tests to establish their baseline level, and then set different game difficulty according to their attention deficit level, let them participate in similar attention games. In the game, the performance of attention will be positively feedback from the game, which will prompt the child to continuously improve his or her concentration ability. This is actually a kind of neurofeedback training that works well in brain science and behavioral science.
Another interesting direction is to combine EEG with VR.
The potential benefit of combining with VR is that non-medical EEG devices can still produce less differentiated output due to accuracy issues, which is why Perri Karyal later needed to add devices such as eye tracking and gyroscopes to her EEG devices. VR devices come with these capabilities themselves, and multimodal inputs may make the gaming experience better.
The prospect of VR combined with steady-state vision-evoked potential-triggered EEG is particularly interesting. One weakness of steady-state visual potential-triggered EEG is that the success rate of visual triggering is reduced in bright light environments. Through the combination with VR, this limitation is greatly weakened. NextMind, a French startup that has been acquired by Snap, offered one such VR+EEG device in 2020. In its demo game, Mindvasion, players can put on VR devices and directly visually aim and kill enemies.
04 AI, a new direction for brain-computer?
According to the brain-computer interface industry research and analysis report proposed by China Renaissance Capital in 2022, invasive brain-computer interfaces are mainly used for clinical-level applications, and specific clinical indications are more inclined to some refractory brain diseases such as epilepsy and limb movement disorders. Non-invasive brain-computer interface, clinical-grade applications include consciousness monitoring, improvement of consciousness disorders, improvement of cognitive impairment, improvement of mental disability, improvement of ADHD in children, replacement of sensory deficits such as hearing, vision, and touch. Typical consumer-grade applications include sleep monitoring, early monitoring of depressed mood, recovery, stress and mood monitoring, and more.
"The rigid demand attribute of the market determines that brain-computer interfaces are mostly used in brain science research and rehabilitation of patients with brain diseases at this stage." Sun Rong, a senior analyst in the medical industry at the Sullivan Head Leopard Research Institute, said.
Obviously, the scene in the field of medical and health, unlike the mass entertainment scene such as games, is not so demanding for the comfort of hardware, the time and price of personalized adaptation, and it is also the biggest opportunity for entrepreneurs at present.
However, with the development of hardware, the direction of entertainment must be an important scene of brain-computer interface. According to a report released by the Shanghai Artificial Intelligence Industry Association in 2021, the global generalized brain-computer interface market is expected to reach more than $330 billion in the next five years (2021-2025), with the most important areas being in healthcare, gaming and entertainment, and education technology.
The relevant person in charge of Qiangbrain Technology said in an interview: "Although we currently have no intention of developing into the field of pure games, I believe that game entertainment as an application scenario of the second brain-computer interface in addition to medical rehabilitation has a very broad prospect."
Artificial intelligence may also be a change in the industry.
In 2022, Meta announced the results of a study called "Brain Signal Reading". The researchers trained the AI using the data sets corresponding to statements and brain waves, greatly improving the ability to infer language signals from experimental subjects through EEG and MEP, another non-invasive brain-computer interface technology.
Therefore, in theory, even if the EEG device does not have a large breakthrough in hardware, the combination of artificial intelligence and EEG technology, from the "software", the accuracy of EEG equipment may be greatly improved.
Although it is a little far away, in the long run, the breakthrough of brain-controlled games may also come from invasive brain-computer interface technology, or the advancement of other non-invasive brain-computer interface technology, such as magnetoencephalography, functional magnetic resonance, functional near-infrared optical brain imaging, positron emission computed tomography, etc.
At the 2023 Zhongguancun Forum that ended on the afternoon of May 30, Zhao Zhiguo, chief engineer of the Ministry of Industry and Information Technology, revealed that the Ministry of Industry and Information Technology regards brain-computer interface as an important direction for cultivating future industrial development, and should strengthen the exploration of brain-computer interface application scenarios to accelerate the vigorous development of the brain-computer interface industry.
In the case of good policies, whether entrepreneurs can find a better direction and intelligently bypass or solve the barriers to commercialization of existing brain-controlled games is unknown, but it is worth looking forward to.
This article is from Geek Park