When it comes to motion games, most players are likely to think of the Nintendo Wii console that has sold more than 100 million. Before the Wii became popular around the world, the most successful motion gaming device was the PS2's camera, EyeToy. Looking back on history, EyeToy is also the first somatosensory gaming device to exceed 10 million sales. 20 years ago, on July 4, Sony lit up the dawn of somatosensory games with EyeToy, but handed over the noon of the red sun to Nintendo, a history that is thought-provoking.
Young talent
Richard Marks is not well known in the gaming industry, but the genius doctor led the development of a number of console peripherals such as EyeToy, PS Move and PSVR, and can be described as the three dynasties of Sony. Max was born in Michigan in 1969. Max has been a mathematical genius since childhood, and when the school was equipped with the first computer in high school, when the teacher did not know how to operate the computer, Max had already taught himself programming knowledge.
After graduating from high school, Max was admitted to the computer science department at the Massachusetts Institute of Technology. During the summer of his freshman year, Max came to an internship at a space company, and this experience made him know that computer scientists are only in an auxiliary position in the field of aerospace engineering, and the leader is engineers, so Max turned to aerospace electronics.
Richard Marks is an expert in camera technology
After graduating from university, Max entered the Institute of Space Robotics at Stanford University in California to continue his studies. Initially, Max worked at Stanford on space shuttle robotic arms and remote controls for space vehicles, which he thought were interesting, but the space industry was very expensive, with limited rocket launches and few job opportunities. Max later interned at the Monterey Bay Aquarium during the summer, where he helped design an underwater robot that could track fish and map the ocean floor, which was far less technical than a space car, but gave Max a real sense of accomplishment.
The 26-year-old Max received his doctorate with the thesis "Automatic control of underwater robots by visual sensors". Max believes that the key to improving robot control is to optimize the camera's data algorithm and let the program automatically adjust the robot according to the image, which is more efficient and less expensive than simply improving the control system. According to the thesis plan, Max installed a pair of cameras for the robot at the Monterey Bay Aquarium, and through the synthesis of 3D images by two cameras, the robot can analyze underwater terrain and currents, and automatically track the movements of fish schools, which is far more efficient than human remote control. The system developed by Max is still widely used by underwater robots today.
Underwater robot developed by Max
After earning his Ph.D., Max joined a startup that studies cameras, developing software that lets cameras automatically track the movement of people for videoconferencing. After a year and a half, the startup was acquired by a Silicon Valley company, and Max became a consultant in Silicon Valley.
When the parent company on the Silicon Valley side encounters a problem that it can't solve, they will ask Max, that's all, the rest of the process has nothing to do with Max. It's a job with more money and less work, but Max is bored. He felt that if he simply wanted to make money, he could go to Wall Street to pan for gold, and he wanted to continue to develop products so that he could get a sense of accomplishment.
Max's parents once opened a small shop selling games in the early 80s of the 20th century, and when he was a teenager, Max often worked in the store, although the small store closed down in the tide of "Atari shock", the game industry still left a lot of good memories for Max, and he hopes to develop hardware for the game industry.
The turning point came in March 1999, Sony publicly demonstrated the PS2 at the Game Developers Conference (GDC) in San Jose, California, Richard Marks came up with the idea of developing a camera motion game for the PS2, and then he joined Sony's research institute in Foster, California, and the idea of motion game was selected by Phil Harrison, vice president of technology research and development at Sony Electronic Entertainment (SCE).
Phil Harrison, who combines technology and market capabilities
Harrison believes that somatosensory games have a heavy responsibility to expand the number of players. The complexity of traditional controllers has approached its limit, and the PS2 controller has a total of 16 buttons, and casual players holding such a complex controller in their hands is like holding a grenade with a bolt unplugged, they will only throw the handle away. The graphics of the game have improved rapidly over the decades, but the controller has not changed essentially, and the developers have burned a lot of money for the game's graphics and music, but the sales have been limited by the controller. In Harrison's view, only junior high school students can study and understand such a complex controller, younger players can't understand it, and older players don't have time. The development cost of games is getting higher and higher, but the controller limits the room for game sales to increase, which is not reasonable, and somatosensory games can break this situation.
Richard Marks became a Sony employee for 19 years. Max laughed that he never thought he would work at Sony for so long, and the reason why he chose Sony was mainly because Sony had sufficient R&D funds and perfect market channels. Some people suggested that Max set up a startup to study somatosensory technology and master the patent himself to make more money, but Max did not agree, he knew that a product needs capital and market support to succeed, and the odds of a startup winning are not great. As for why he worked for 19 years, he had the idea of leaving Sony several times, but every few years Sony would launch new consoles to provide opportunities for new peripherals, and these research and development work made him feel interesting and fulfilling.
Various third-party companies have launched several failed somatosensory peripherals for FC and MD consoles in the 90s of the 20th century, all of which have been lost in history. Somatosensory technology did not mature until the late 90s, and Yuji Naka's team designed a handle for DC consoles that combined remote control and light gun functions, similar to the later Wii Zapper. The handle was vetoed by Sega, but DC has since introduced a variety of somatosensory peripherals, such as a fishing rod for Sega Bass Fishing in 1999 and a sand hammer for Samba de Amigo in 2000. The accuracy of DC somatosensory peripherals is not as good as that of the later Wii, and only tens of thousands of copies have been sold, but they have achieved technical success.
Hiroshi Naka's team designed the DC controller
Nintendo veteran Katsuya Nakagawa, who participated in the development of multiple consoles such as FC, established the "New Generation" company in 1995. Nakagawa said Nintendo was obsessed with advanced technology in the 90s but ignored the nature of gaming as a family entertainment. He devoted himself to developing highly integrated all-in-one chips for five years, and launched a number of motion sensing devices from 2000 to 2002 by contacting game companies such as Bandai and Konami, which were both toy businesses. Compact, plug-and-play, these products come with a bat or racket that detects changes in sensation through infrared technology. Despite the simple picture of the all-in-one machine, with intuitive physical pleasure, Nakagawa's products have accumulated more than 100,000 sales in 3 years.
A somatosensory baseball game in the form of an all-in-one
The early products mentioned above may give today's gamers a sense of being out of the world, and Nintendo's Wii console is the first motion device that most people touch. The Wii's somatosensory technology originated from pilot and engineer Tom Quinn, who successfully registered the patent in the United States in 1999 and tried to sell it to a game company in 2001, but hit a wall twice.
Steve Ballmer, then CEO of Microsoft, met with Tom Quinn, who assumed that motion sensing technology was promising and recommended Quinn to the Xbox console team to discuss details. However, the Xbox team was very arrogant about Quinn, and the team said that Microsoft could develop a better somatosensory solution internally without Quinn's patent. Quinn then tried to sell the patent to Sony, and the "father of PS" Ken Kutaragi answered more clearly: "The cost of this technology cannot be reduced to 50 yen, and Sony will not consider it." ”
Tom Quinn was not optimistic about Nintendo's prospects at the time, but he could only choose Nintendo after hitting a wall twice. In early September 2001, the NGC console had just ushered in the failed Japanese version of the debut, and Tom Quinn visited Nintendo during this eventful time. Yamauchiura, the old president at the time, had prepared himself to step back into the background, and Nintendo's major events were discussed by six senior executives of the board of directors. After some heated discussion, Atsushi Asada, then vice president of Nintendo, purchased a motion sensation patent. This technology was only used in reserve in 2001, and it was not until 2004 that new president Satoshi Iwata decided that NGC's successor console would abandon high-definition graphics and choose the somatosensory route, and the Wii's debut would not wait until 2006.
Tough search
Closer to home, the camera developed by Richard Marks for the PS2 was originally called iToy (interactive Toy), and later Phil Harrison renamed it EyeToy (meaning "visual toy"), a concise name that was also endorsed by the marketing department.
Max's initial test program for EyeToy was based on color detection, using a tennis ball as a detection marker to make a virtual 3D object stand on the tennis ball, and the object would move by inertia as the tennis ball moved, and then Max replaced the tennis ball with a pink ball used by the Sony Aibo robot dog. Max sticks a pink ball into the tip of a pencil and waves the pencil in front of the camera, and the pencil in the screen turns into a magic wand that releases magic.
EyeToy's initial test of tennis
Powder balls that come with the robot dog Aibo
In the fall of 2000, Phil Harrison flew Richard Marks to London, England, to give a lecture to show the magic wand program in front of all the developers of Sony's London studio, and a group of interested developers volunteered to make games for EyeToy, and then the London studio discussed the details.
At that time, EyeToy's image recognition algorithm was very rough, and the magic wand had a noticeable operation delay on the screen, and it was impossible to recognize subtle movements. From the perspective of the market, EyeToy can guarantee sales if it launches a "Harry Potter" game, but at that time, the "Harry Potter" game adaptation rights belonged to EA, and Sony London Studio did not want to negotiate with EA around this matter. In order to speed up development, the London studio advised Max to temporarily abandon the magic wand game of color recognition, which would be much less technically difficult if the motion detection algorithm scanned the player's full body outline in front of the screen. With this in mind, the London studio produced EyeToy's debut game, EyeToy: Play, a collection of mini-games that showcased EyeToy's gameplay and potential.
Many people told Max that they were not optimistic about the future of EyeToy, this kind of somatosensory peripheral has never been sold on consoles before, once the initial sales of peripheral peripherals fail, follow-up game support is impossible to talk about. In order to ensure the success of EyeToy's debut, Max spent a lot of thought on cost, and the bundle of the EyeToy camera and the "EyeToy: Play" disc cost only $49. Max had experimented with other somatosensory schemes, either the software algorithm still needed to be polished, or the hardware cost was high, and "EyeToy: Play" was already the optimal solution at that time.
Strictly speaking, Max is not a traditional game developer, he is mainly responsible for the development of software and hardware tools to provide solutions for others, the London studio is responsible for the actual game development, and Max provides technical support. Although Musk also went to London, England for 3 months at a critical moment in the summer of 2001, most of the time he spent solving problems remotely through the Internet in Foster, California.
EyeToy can read the player's whole body movement and attach it to a 3D model, but the London studio believes that it is more impactful for players to see their image on TV, so EyeToy: Play is the protagonist of the player, not a virtual 3D model. At that time, the software recognition algorithm was difficult to exclude the noise in the background, only suitable for detecting the action of opening and closing, and the player's physical exertion was fast, suitable for short and sharp mini-games, so "EyeToy: Play" became a collection of mini-games.
The player appears directly in the screen
The London studio initially developed 30 mini-games, from which 12 were selected after testing. SCEA in the United States has well-established testing teams in Foster City, California and Santa Monica, while SCEE in Europe lacks such teams and can only outsource testing. However, there were no well-established third-party game testing agencies in Europe at that time, only office software and web testing companies, which could only report how many bugs the software had, and could not report the gameplayability of the game. The London studio had to find a web testing company to go through the process and recruit a large number of casual players to participate in the test.
Because casual gamers didn't know much about the game's menu, most people got stuck on the Create Save menu in the first round of testing, and the developers had to simplify the menu steps. After that, the test ran into the problem that some of the special effects were not completed, because EyeToy: Play is a rare innovation, and many gameplay methods are not intuitive until the special effects are completed. Phil Harrison said: "Sony's marketing department couldn't read the game until the picture was completely completed. When the game reaches 50% completion, it looks like it's only 1%. When the game reaches 80% completion, it looks like only 2%, and the rest is up to your own imagination. ”
Regarding the choice of mini-games, Harrison also has a set of criteria, if a gameplay experience on the controller is already perfect, then there is no need to convert it into a somatosensory feel. On the contrary, the game of rubbing glass is meaningless on the controller, but it is quite fun to replace it with a body feeling.
Glass wiping mini-game
The average PS2 game costs between $3 million and $5 million to develop, and EyeToy: Play costs more, though Harrison thinks it's worth the money. On the one hand, SCEE has established its own test team for this work; On the other hand, a lot of development money is spent on improving EyeToy's recognition algorithm; These investments will pay off for SCEE in the long term.
Prepare for the launch
As for how many copies of "EyeToy: Play" will sell, the developers themselves have no idea, and the London studio's internal predictions can be described as polarized: "Either less than 100,000, or more than a million." "This new concept game may fade out, it may emerge and sell successfully, and no one knows."
At that time, the largest game show in Europe was the European Computer Trade Show (ECTS) in London, and with the mentality of giving it a try, SCEE took EyeToy: Play to ECTS in August 2002, and the results were rave reviews. In past exhibitions, it was often the children who were enthusiastic in front of the demo machine and the parents sat next to them and chatted. This time, the child's parents and even grandparents joined the game. One of the children even had fun with his grandmother in a wheelchair, an unforgettable scene for the London studio, and they believed that EyeToy: Play would surpass millions of sales.
ECTS was nowhere near as influential as E3, and EyeToy: Play attracted more attention at E3 in May 2003, and even Nintendo's "Father of Mario" Shigeru Miyamoto played the game at E3. The European release date of EyeToy: Play is set for July 4, 2003, November 4 for the US version, and February 11, 2004 for the Japanese version. European sales exceeded the 2 million mark in only 4 months. At the time, cameras were not common peripherals for PCs, and Logitech, which produced cameras, set a new revenue record in the fall of 2003 and made a fortune with EyeToy.
Logitech's EyeToy camera
Richard Marks said that at that time, ordinary PC cameras focused on improving clarity, and the refresh rate was often only 10 frames. EyeToy's resolution of 640×480 is not high, but it provides a refresh rate of 60 frames, which is naturally designed to reduce the latency of game operations. If you reduce the resolution of EyeToy further to 320×240, the refresh rate can even be increased to 120 frames.
Released in Europe on November 14, 2003, EyeToy: Dance was the second EyeToy game, and this dance game was far less sensitive to light detection than EyeToy: Play. Regardless, EyeToy sold 2 million in Europe in 2003 with these two games alone.
EyeToy sold 400,000 in the U.S. in 2003 and 160,000 in Japan in 2004, a far cry from Europe. On the one hand, the launch time of the American version and the Japanese version of EyeToy are adjacent to a number of masterpieces, which affects sales. On the other hand, Sony's marketing department in the United States and Japan has never promoted EyeToy.
SCEA in the United States is keen to promote all kinds of core masterpieces, and lacks interest in casual games. The Japanese side of SCEJ actually said that "Japanese players do not like this kind of casual game". Ironically, the all-in-one game "Dragon Quest: Sword God" developed by the "New Generation" company reached 500,000 cumulative sales in Japan in 2004, and the motion game has a great future in Japan, and the arrogance of SCEJ's marketing department is simply unreasonable.
All-in-one machine "Dragon Quest: Sword God"
2004 was a mixed year for Phil Harrison, and although EyeToy's sales in the U.S. and Japan were ruined by careless marketing departments, sales of the camera steadily climbed in Europe, and third-party companies that took the wind provided more support for EyeToy. Most third-party games' support for EyeToy is limited to shooting headstickers, and only a few third-party companies have shown sincerity. Although Sega Superstar is a collection of mini-games, it combines the gameplay and body feel of various Sega classic games, so that veteran players can also enjoy it. Harmonix, known for its music games, forged ahead and developed a sci-fi game, EyeToy: Anti-Gravity Skateboarding, which won critical acclaim.
With the support of Phil Harrison and riding on the wave of EyeToy, SCEE continues to tap the potential of peripheral casual gaming. The 2004 microphone game "Singer" and the 2005 variety show answerer game "Buzz" were successful, and their sequels even extended all the way to the middle of the PS3 era many years later.
Speedy EyeToy: Anti-Gravity Skateboarding
"Buzz" bundled with answerers
Multi-party dogfights
Katsuya Nakagawa's "new generation" company launched XaviX in 2004 with interchangeable games. It's a fitness console that costs $79 for the base body and $59 for the game, many of which require specific peripheral controllers. Losing the brand effect of a large manufacturer, this product responds to few people. After that, the fitness equipment launched by the "new generation" was also unsuccessful, and after years of hard support, the "new generation" finally declared bankruptcy in 2022.
Nintendo's new console Wii, which was first released in 2006, achieved somatosensory operation through accelerometers, and the cost of accelerometers had been greatly reduced, but Richard Marks believed that a single accelerometer could not guarantee somatosensory accuracy, and did not follow up on this solution.
Max introduced the EyeToy successor, PS Eye, for the PS3 in 2007. In terms of resolution and refresh rate, PS Eye has not improved, Max focuses on other improvements. PS Eye is more sensitive to light and works properly with the light turned off only by the light source of the TV. In addition, PS Eye's microphone matrix allows players to achieve clear voice chat without wearing headphones. However, SCEJ's PS Eye card game Eye of Judgment failed, causing the camera to fall into a temporary silence.
PS3's camera PS Eye
At this time, Sony's PS3 console was in a difficult battle because of the high price and cost, Phil Harrison believed that the Wii was just an enhanced version of NGC with its own somatosensory feeling, so Sony only needed to bundle EyeToy for the PS2 to greatly ease the Wii's offensive. Sony executives in the United States and Japan both hope to popularize the PS3 as soon as possible at any cost, and vetoed Harrison's proposal, and Harrison could only let SCEE develop a few more PS2 games.
In 2008, the London studio released the last two new titles for EyeToy, EyeToy Play: Heroes, and EyeToy Play: Cheerleading, the former with a plastic sword and the latter with two flower balls. Richard Marks perfected EyeToy's color recognition algorithm as early as late 2003, and through these simple plastic props, EyeToy can achieve Wii-like somatosensory play.
Simple and effective plastic props
Nintendo used the Wii Motion Plus patch in 2009 to equip the Wii with a gyroscope to improve the accuracy of operation. Sony's PS Move, released in 2010, combines a gyroscope, accelerometer and magnetometer to achieve the highest accuracy and lowest latency. Players will need PS Camera to capture PS Move's data, and the set for both devices and the accompanying game costs $99.
Many players suspect that Max imitated the Wii's ideas, but Max said that PS Move was originally developed for PS2 in 2003, when the cost of various parts was high, the price of the set would be as high as $149, as a casual gaming peripheral, such a price has no future, so Sony hid PS Move for 7 years.
PS Move for use with PS Eye
Max believes that the Wii is a very successful console, and although the Wii's somatosensory solution is not technologically advanced, Nintendo has managed to adapt to local conditions and develop excellent somatosensory games with limited technology. As for Microsoft's Kinect camera launched in 2010, Max respects all attempts by his peers in principle, but he doesn't think the Kinect is going in the right direction. Max tried an infrared camera like the Kinect years ago, but he abandoned it because it was difficult to balance cost, accuracy and latency. The Kinect has a resolution of 640×480 and a refresh rate of just 30 frames, resulting in a noticeable operational delay.
PS Move surpassed 15 million sales in 2012, and Kinect surpassed 20 million in 2012. PS Move sales failed to meet Sony's expectations, and Kinect completed Microsoft's initial sales targets. However, the sales-only theory is one-sided. Sony did not mandate first-party studios to support PS Move, but Microsoft closed a number of first-party studios that were reluctant to develop Kinect games, a decision that weakened Microsoft's development power. From that perspective, the Kinect is the real failure.
In 2013, the new camera PS Camera was released with the PS4, this time Max used the stereoscopic imaging technology he studied during his doctoral studies, and synthesized depth data through two 1280×800 cameras. The 2016 PSVR was paired with PS Camera and PS Move to form a complete solution. As early as the era of 3D glasses of PS3, Max believed that the combination of somatosensory and stereoscopic picture was a new direction, and the cooperation between PSVR and PS Move was a natural evolution of this direction.
Peripheral combination for PSVR
After working for Sony for 19 years, Richard Marks moved to Google in 2018, and his development results have not yet been made public. The PSVR2 in 2023, as a virtual helmet for PS5, is no longer compatible with peripherals previously developed by Max.
Phil Harrison left Sony in 2008 and has since worked for Infogames, Gaikai, Microsoft and Google, failing to achieve new results in the past 15 years. After Google shut down its Stadia cloud gaming business in 2023, Harrison disappeared from the public eye.
Still, looking back, EyeToy's achievements ahead of its time are still a glorious moment for Max and Harrison. The final cumulative sales of EyeToy is 12 million, which is the first somatosensory game device to exceed 10 million sales. As the dawn of a new era, EyeToy will be remembered by history.