As we all know, the metaverse and the game industry play an important role in the XR field, and its industry scale is also increasing day by day. However, according to Goldman Sachs' report as early as 2016, XR technology is expected to improve some markets, including healthcare, in the coming years.
Six years later, that prediction has come true, and XR technology has become a common use in the medical field. It has proven to be of great help to both doctors and patients: patients can learn about their pathology through XR, and doctors can perform high-precision and complex surgeries with XR. In the eyes of industry insiders, XR is changing people's lives in unimaginable ways.
It is no exaggeration to say that AR has demonstrated the future of medical technology to some extent.
1. XR's potential and advantages in the medical field
MarketsandMarkets analyzed ar/VR technology growth in healthcare from 2017 to 2025 (expected). Company data predicts that the consolidated annual growth rate will reach 30.7% over the time frame studied.
As mentioned above, VR and AR have a lot of potential in healthcare, but how will people use it?
For now, XR is bound to make it easier to plan surgery – in the past, even the most experienced surgeons sometimes experienced unexpected conditions when operating on patients, and the immediate response determined the final outcome of the operation. In the future, the intervention of virtual technology will greatly reduce accidents and effectively reduce the risk of surgery.
In addition, the researchers also looked at ways to use VR to educate patients before surgery. For example, a person can see a digital version of their brain, and the problems that surgeons need to fix and how to achieve them.
At the same time, virtual technology can improve medical school training: Johns Hopkins University has set out to build an AR project that allows medical students to convert their anatomy applications into AR interfaces. The augmented reality tool shows an internal view of the body above the student's body, and it also features an interactive control program that is extremely sensitive to real gestures.
Application of XR in surgery
Computer-aided AR, also known as Computer-aided Drawing, is a mapping tool that allows users to make accurate data models using AR.
At present, professional institutions have conducted a review study on the past, present and future application of computer-assisted AR technology in surgery. From a categorical point of view, in different types of surgery, AR can be used as a display or model. In related medical training, such as echocardiography training, laparoscopic surgery training, neurosurgery training, etc., AR technology is also applied and has a very broad prospect.
In general, surgeons are usually the first to adopt technical tools that enhance the surgery and patient experience among physicians in various departments. Applications include Operative Benefits, Broadcasting and Recording of Surgery, anatomical assessment, remote guidance, and medical education.
Application of XR in ophthalmology
Currently, an ophthalmic AR environment has been developed. The environmental system is used for more precise laser treatment of ophthalmic diseases, telemedicine and real-time image analysis, measurement and comparison.
This environmental system is designed around a standard slit lamp. A camera connected to a slit lamp captures the image, which is then sent to a video acquisition board. Next, the image is processed using a computer workstation and a fast registration algorithm is applied to it. The computer's processing results are ultimately output to a display with adjustable brightness and contrast, which is attached to the eyepiece of the slit lamp.
Design the intracranial implant with XR
Scharver C et al. conducted this aspect of the study. They say that the haptic AR environment can convey the feeling of touch to the user. Specifically, it is to obtain data from the patient's CT and create a virtual 3D skull model, thereby superimposing the model into reality. Medical skull sculptors can better feel and observe models with this technique, which is very helpful for improving the quality and speed of their work.
The Personal Augmented Reality Immersive System (PARIS), a new prototype display system, will also be used in conjunction with the above model. The addition of this system can make the skull engraver feel like holding a 3D tool in his hand. Neurosurgeons, paleontologists and radiologists have all shown great interest in the system and would very much like the opportunity to try it out.
At present, industry experts have studied this. They mention that in rehabilitation-related tasks, the system can create a virtual audio and visual experience by tracking the subject's hands, including the movements of the elbows, shoulders and wrists. Real-time data and photos are also sent to the doctor for further evaluation. Laboratory applications have demonstrated the availability of the technology.
They also say the system can incorporate real objects into the mission as needed. It also controls external objects, ensuring patient safety and comfort. In addition, another advantage offered by the system is that the therapist can modify the task according to the needs of the patient. They have successfully applied this in their stroke rehabilitation exercise program.
Design a hand and arm rehabilitation system with XR
According to authoritative institutions, top research institutions have published papers on this aspect: they first evaluated the AR system on an abdominal model, and then on patients in the operating room. The preoperative images of the patient and the needle positions performed by the doctor are recorded in a coordinate system. Feature points are extracted and processed by validation algorithms.
The results show that in less than a minute of insertion time, the predetermined goal of 2 mm accuracy can be achieved. The results inspire confidence that the AR system will make a difference in the field by providing accurate guidance and information during the patient's full respiratory cycle.
2. The development trend of XR medical treatment
In general, the research and application of XR medical treatment has gradually increased in recent years, and the number of papers has also shown an upward trend. Martin Eckert et al. found through meta-analysis that of the 338 papers, 84.3% (n=285) were AR projects dealing with patients' actual treatment, and 15.7% (n=53) were involved in training scenarios. Of the 285 papers identified as belonging to the therapeutic category, 69.5% (n=198) of the projects involved operating room scenarios and 9.1% (n=26) were set in a rehabilitation setting. In addition, 21.4% (n=61) were directly involved in the treatment of patients.
The covid-19 pandemic has caused some businesses and schools to close, cause economic damage and a large number of deaths. In the face of the crisis, the scientific community and related industries are developing various technical methods that can benefit society, and the application of AR technology is one of them. For example, controlling social distancing by helping individuals maintain a 2m distance from others through AR applications; helping medical students learn the human body in 3D through online education in AR applications; improving the way patients see patients and doctors through AR telemedicine. In these aspects, there are relevant cases to prove the effectiveness.
It is foreseeable that AR technology will affect the treatment of patients in the future. With the optimization of hardware and software technology, AR will be better integrated into existing workflows and create opportunities for patients, doctors and relevant industry personnel.
3. Top XR companies in healthcare
As described in this article, the promise of medical augmented reality and virtual reality is compelling, and the current results are enough to change the way many people think about the medical landscape, and it can make more people look forward to the upcoming medical approach. According to Sina VR statistics, nearly a thousand companies around the world have been involved in the XR medical field. In terms of scale and achievements, the attention of the following ten companies has become the highest in the industry:
1、MindMaze
Swiss company MindMaze was founded in 2012 and raised more than $108.5 million. The company's mission is to improve human recovery, learning, and adaptability after neurological diseases. The company has been researching and exploring the intersection of neuroscience, biosensing, engineering, mixed reality and artificial intelligence for more than a decade, and its product MindMotion GO has now received FDA certification and CE certification.
In September 2020, MindMaze and Mount Sinai Healthcare in New York announced a home-based study of remote neurological rehabilitation that would assist stroke patients in continuing their home rehabilitation training with the virtual support of clinicians. MindMaze Labs, the company's research and development arm, is currently exploring other areas beyond healthcare, such as bringing breakthrough neuroscience technologies into everyday life and providing technical support for the next generation of human-machine interfaces.
2、Touch Surgery
Founded in 2012 and based in the Uk, Touch Surgery has raised more than $91.1 million. The company's research products are designed to harness the power of AI, data, and visualization to save lives. Touch Surgery, founded by two British surgeons, was acquired as a subsidiary by Medtronic in 2020. The company's team consists mainly of visual artists, software, hardware and AI engineers, data and medical scientists, and surgeons.
Touch Surgery's interactive surgical simulator provides a practical guide to surgery and medical procedures, enabling physicians to quickly master the surgical process, test proficiency, and train in surgical simulations. With 3 million global users, the company's product Touch Surgery Professional is the first surgical video recorder equipped with real-time AI and provides a secure cloud storage solution for surgeons' surgical videos, and is currently promoting the development of new products that provide real-time dynamic AI for operating rooms.
3、Vicarious Surgical
U.S. company Vicarious Surgical was founded in 2014 and raised $30 million. The company is committed to improving the quality of life of patients, enhancing the level of surgeon surgery, and expects to provide high-quality medical services worldwide. The company's co-founders, Adam Sachs and Sammy Halifa, began to develop VR technology when they were undergraduates at MIT, and later combined the idea of laparoscopic surgery to determine the company's development direction for the joint application and product landing of "surgical robot + virtual reality".
Surgeons operating on patients by remotely controlling the movements of small abdominal surgical robots through VR technology is the dream of founder Sachs, the company's Vicarious Surgical Inc. The surgical system received the title of "Breakthrough Device" by the U.S. FDA in 2019.
4、Augmedics
Founded in 2014 and based in Israel, Augmedics has raised $25.1 million to date. The company, whose headsets are FDA-licensed, began selling their headsets nationwide earlier this year, starting with spinal surgery. The company's first surgical augmented reality operating system, "xvision," allows surgeons to see patients' internal anatomy through skin and tissue as if they had X-ray vision.
Current research data from the company shows that surgeons are 98.9 percent accurate when using VR headsets when placing spine screws in the body.
5、Health Scholars
Founded in 2017, Health Scholars has raised $17 million. The company released two VR games for emergency resuscitation training in 2019, and the solutions for its products were designed by clinicians to ensure that VR medical simulation products can meet healthcare certification standards and professional association guidelines. By learning from this product, doctors can spend less time mastering medical technology and improving resource utilization.
Currently, its VR medical simulation products focus on emergency nursing training, helping medical learners to accurately master first aid skills by providing immersive visualization and interaction systems immersed in surreality, so that medical students and nurses can master general nursing, perioperative and obstetric operations, and emergency treatment of adults and children before entering the hospital.
6、Limbix
Limbix, a VR company that provides mental health treatment to young people, is headquartered in California and has raised €14.5 million. Depression and suicide in adolescents are on the rise, but expensive treatment costs and appointment wait times limit time for timely treatment. The company's team is a diverse team of clinicians, product designers, researchers, and engineers, and its product, Limbix Spark, is currently the first fda-approved clinical VR treatment for adolescent depression.
Spark's long-term plan, based on cognitive behavioral therapy, focuses patients on completing activities of value and interest that stimulate a sense of pleasure and, in turn, improve depressive states. In the coming years, Limbix will build on Spark's R&D with more trials and work to get Limbix Spark approved by the FDA as a Class II medical device.
7、Osso VR
Osso VR, based in Boston, Massachusetts, was founded in 2016 with $16 million in funding. Currently, there is a problem of inefficient training methods for residents and surgeons, with data showing that 30% of surgical graduates are unable to perform surgeries independently. Compared to advances in technology, our surgical training system has not changed for more than a century.
Justin Barad, co-founder and CEO of Osso VR, a Ph.D. in bioengineering and orthopedic surgeon, founded VR Medical Training and plans to use his own technical background to solve current medical training problems with a mission to improve patient safety and popularize modern surgical techniques.
Osso VR uses VR technology to provide training platforms, content, and tools that fill the gaps in surgical training. Today, Osso VR has partnered with many of the world's top orthopedic hospitals and equipment companies. The VR headset it developed to train orthopedic surgeons by simulating 1:1 movements during surgery with haptic feedback has included training on tibia nails and hip and knee replacement models. Osso VR was selected by the U.S. Department of Education as a top educational simulation company.
8、Oxford VR
Uk-based company Oxford VR, which raises €13.2 million, focuses on the treatment and intervention of clinical mental illness. Founder Daniel, a professor of clinical psychology at the University of Oxford, has developed psychological interventions based on mature treatment options to treat patients using immersive VR technology.
Oxford VR develops treatments that are fully automated, enabling patients to access high-quality treatment more conveniently and economically through virtual coaching. For example, fear of heights is a common clinical psychological problem, estimated to affect 3-5% of the population, studies have proved that immersive VR therapy can successfully reduce the patient's fear by an average of 68% in just two hours of treatment, which proves the great potential of VR treatment.
In addition, the company's developed VR product to increase social engagement is a breakthrough mental health intervention that uses VR technology to help individuals overcome social avoidance emotions of anxiety, including phobias, panic disorders, social anxiety disorders, depression, and schizophrenia.
9、Psious
The Spanish company Psious raised about $15 million to focus on providing VR treatments for patients with poor mental conditions, such as fear of flying, injections, various animals, public speaking, and anxiety or fear. With the help of VR devices and apps, patients enter an environment that is very frightening to them with the help of a doctor.
The patient's task is to confront his fears and gradually untie the knot. The company has developed more than 70 virtual environments and dozens of features to intervene in anxiety disorders, specific phobias, eating disorders, obsessive-compulsive disorders, and addictive disorders, including through relaxation techniques, mindfulness therapy, and attention training.
During patient training, dozens of parameters can be interactively customized, as well as real-time monitoring responses can be achieved to obtain information on the patient's condition, and all patients' treatment course data can be saved to monitor their improvement.
10、Surgical Theater
When founders Moty Avisar and Alon Geri created The Musical Theater, they imagined surgeons being able to preview their procedures like pilots, as fighter pilots could pre-execute surgical plans in the same way that they could pre-execute their missions, and could use these simulators to help patients participate in their care plans.
Neurosurgeons in dozens of medical and academic institutions can already educate patients and engage patients in virtual surgical environments using the immersive experience of the Medical Virtual Reality platform developed by The Musical Theater, giving patients a 360-degree experience – walking into their own anatomy , while also providing physicians with the opportunity to simulate surgical plans.
Current customers include UCLA, New York University, University Hospital Case Medicine Center, Mayo Clinic, Mount Sinai Medical Center, and Stanford University.
4. Our views
In the author's opinion, VR/AR medical treatment is not a technology-driven track, and hard power is the core factor that determines the ultimate success or failure. No matter how innovative the entrepreneur's idea is, its solution must bring significant results in practice, and what the investor values is the operability of the product landing. Therefore, after 2016, known as the "first year of VR", companies in this field have focused more on research and development and polishing technical products.
In addition, VR/AR medical treatment as a pure digital technology, if promoted and sold as a medical device product, it also requires strong financial strength, lengthy product registration and clinical trials. Obviously, this is not what every company and every investor can afford. Because of this, many companies in the industry have chosen the direction of more easy to obtain results (benefits), such as cutting into medical education, surgical simulation, etc., so a number of excellent VR medical education and VR surgical simulation training companies have emerged.
At present, although the domestic VR medical track is not as large as the scale of foreign financing, our technical strength cannot be underestimated, such as medical WeChat, Palm Network Technology, Touch Illusion Technology and Dragon Technology and other companies have also developed rapidly. However, although many of the products of these manufacturers have landed, they still face the problem of difficult to popularize. In our view, this is related to the high cost of related equipment, insufficient market recognition and insufficient sales channels.
As XR's leading media, we expect that more medical products with VR technology can be popularized under the impetus of capital, so that more people can experience the convenience and practicality brought by VR, and at the same time, they can also add a weight to their health.