In the era of the Internet of Everything, it is conceivable that there will be more and more connected devices per capita. But the increase in equipment is not the same as convenient and efficient, device interconnection is not easy, device "smart connection" is more difficult, the most difficult is probably how we can enjoy the convenience of more devices, without being troubled by the cumbersome process of controlling multiple devices, finding and using various applications.
At the recently held Huawei Developer Conference 2021 (together), Huawei gave their thoughts on the above issues. In simple terms, Huawei believes that the way people and devices interact needs to be redefined, and the core weapon is AI. Such an answer doesn't sound unusual, and it can even be said to be a common standard answer. But don't forget, Huawei is building the Hongmeng ecosystem, and based on this ecological base, we see a more "solid" feasibility.
Let's first look at what the redefined way people and devices interact might look like.
Today, health is a proposition that everyone cares about, and many of them manage their bodies through fitness and exercise. Taking the sport of "running" as an example, Huawei envisions an interactive scenario with a "sense of futuristic technology" - under the linkage of smart watches, smart treadmills, mobile phones and other devices, users unconsciously have professional personal fitness coaches, and this is not a real coach, but an AI-based virtual coach.
According to Wang Chenglu, president of Huawei's consumer business AI and wisdom all-scenario business department, this "personal trainer" can formulate the most appropriate exercise plan according to the user's physical state, and will remind the user in real time whether his exercise method is correct during the exercise process, whether it will cause harm to a certain part of the body, how to relax after the exercise, and the appropriate relaxation plan according to the results of the exercise state.
And for this "personal trainer" to be so smart, it needs to get a lot of information. For example, the user's body fat rate, heart rate range, diet and sleep status before exercise, and real-time monitoring of breathing and heart rate during exercise, to understand the situation of exercise such as pace/slope and running posture during running.
And getting more and more accurate information and data means more device collaboration. The more devices collaborate, the more sensors around the user, and these sensors constantly record the user's behavior, so that the "personal trainer" can more accurately capture the user's intentions.
More importantly, in the process of this human-computer interaction, the user does not need to get services and find services around multiple devices, but the device actively understands the user and recommends the corresponding services.
Of course, sports health is just one of the smart scenarios. "I believe that with the blessing of the Hongmeng system, we will have many opportunities to build a variety of scenario-based hyperterminal terminals." Wang Chenglu said.
In fact, today each consumer around the device has a lot of sensors, they can characterize geographical location, spatial information, record physiological indicators, behavioral activities, etc., but the way we use the device is "single device" and "single dimension", and single device, single dimension can not accurately capture consumer intentions, if you can integrate multiple dimensions of information for the scene, you can maximize the capture of consumer intentions.
Therefore, in the case of sensors already existing, the technical problem is how to obtain the sensing signal, how to establish multimodal semantics, and how to match the scene with multimodal semantics to accurately capture the consumer's intentions.
There are three core technology points involved, namely the synchronization of multi-device sensor signals, the annotation of multimodal data, and multimodal adaptation.
The first challenge with multi-device collaboration is complex device heterogeneity. A very important first step in making good use of the sensing signal is to ensure that all sensors report signals in sequence alignment. Due to the different sizes of the equipment, the different sensors, and the different characteristics of the reported signals, it is very challenging to synchronize the timing between the signals.
Hongmeng ecological soft bus technology can solve this problem very well, the goal of the soft bus is to infinitely approximate the ability of the hard bus, and a very important point in its working principle is to have a soft clock. The working principle of soft clock is that when the working device forms a hyperterminal, it captures the clock signal from the crystal oscillator of the strongest device, and once the clock signal is captured, the clock processed by all subsequent services will be based on the soft clock. At present, the accuracy of the soft clock can already be less than 1 millisecond error, Wang Chenglu said that the beta version of harmonyos 3.0 is aimed at 500 microseconds, such accuracy has been able to fully meet the needs of sensor signal synchronization.
The second core technology point is the annotation of multimodal data. Data annotation has always been a great challenge to the direction of AI sample data, so Huawei has proposed an innovative mutual assistance annotation method, which automatically labels data with labeled modes on the basis of clock synchronization and time alignment. In this way, with the more sensing devices, the more devices labeled, the more associated modes will be more and more, and the efficiency of automatic labeling will be greatly improved.
The third core technology point is related to the real-time training of models with small samples. Compared with the traditional large-scale semantic training, the sample data of the sensing signal is very small, and secondly, the sensor is not stable enough to ensure that any device in the hyperterminal is always online and stable, and once the fluctuation occurs, it will have a greater impact on the model. Therefore, the end side can not use the existing traditional AI model to do training, Huawei specially developed a small sample real-time training model on the end side to solve the impact problem caused by small sample data fluctuations and instability.
The core algorithm of the model is sparse coding, which is an unsupervised learning algorithm, the core of which is to find a complete base vector machine, and the vector signal comes in and uses the linear combination of the base vector machine to characterize the input signal. This algorithm can be very effective in avoiding the problem of model fluctuation caused by data instability and sample space instability. This technique is essential if you want to do real-time training with small samples on the end side.
In addition, the sample size may become infinitely larger during the training process, because it is training on the end side, and the hardware resources will naturally be more and more occupied, so that other applications cannot run, so Huawei has also developed an adaptive deployment technology. In simple terms, adaptive deployment can limit the hardware resources consumed by the training model, and automatically make crop adjustments when the model becomes larger to ensure that the model is trained and deployed within the limited hardware resources.
With the above combination of technologies, the sensor's federal data can capture the consumer's intentions in real time and accurately, and the next step is to find the corresponding services according to the intention, and the matching actions are left to the machine to solve, rather than the consumer actively finds, downloads and installs various applications as today. What needs to be done here is to continuously enhance the implicit vectorized intent, find dozens of candidate sets from massive services and applications, coupled with explicitly structured intent, and further screen and logically assemble the candidate sets to achieve accurate feedback on consumer intent.
In fact, harmonyos 2's "little art suggestions" are behind the use of these technologies. Wang Chenglu pointed out that the current use rate of "Xiaoyi Suggestions" is already very high, and it is precisely because Xiaoyi can deeply understand the user's intentions and put the services that may be required into the Xiaoyi Suggestions according to the user's time, place and scene. As a result, users are actively searching less and less frequently.
"If services like 'Xiaoyi Suggestions' continue to persist, there is hope that a unified entrance for human-computer interaction for the Internet of Everything will be created in the future, and xiaoyi can shield a variety of hardware differences." That is to say, in this process, the user does not need to care which device or service they are interacting with, but only needs to interact with the entrance of Xiaoyi, at the same time, due to the intelligence of Xiaoyi, the device itself is also judging the user's intentions. Wang Chenglu called Xiaoyi "the interactive entrance to the future interconnection of all things".
Like harmonyos' design philosophy, Xiaoyi also adopts a complete decoupled architectural design to support elastic deployment. In other words, devices equipped with harmonyos and openharmony can be fitted with the appropriate version of Xiaoyi, regardless of size.
"With a deployment like Xiaoyi, with the interconnection and integration of Hongmeng ecology, in any scene, no matter which device you use at any moment to reach, Xiaoyi can perceive." Xiaoyi will also feedback the information to you according to the way it is reached at this moment, eliminating the process of continuously matching between different devices and finding services for logical assembly. Wang Chenglu said that Huawei hopes to accelerate the arrival of the true era of the Internet of Everything and the Intelligent Connection of All Things through heavy empowerment.
At present, Huawei has provided all capabilities, including Xiaoyi, to partners through software service packages.
This year, harmonyos connect software service package 3.0 added screen devices that support vertical applications, and upgraded the basic service package, enhanced service package, and application service package including Xiaoyi, service center, Changlian and other applications, developers can quickly develop devices with smart AI capabilities by directly calling, and it is expected that by 2022, it will be fully adapted to screen devices that support rich applications.
In addition to having a package, you also need to consider whether it is easy for developers to use. For hardware developers, Huawei provides a one-stop device integration development environment that supports remote development, on-demand customization, one-click compilation and burning, one-click integration service packs and device emulators, lowering the access threshold and development cycle of developers, improving development efficiency, the development cycle of screenless devices has been reduced from 2 months to 2 weeks, and screenless devices are expected to be reduced to less than 2 months in 2023.
Download address: https://device.harmonyos.com/cn/develop/ide
For application developers, Huawei has also launched a one-stop device integration development environment, which provides more than 50 atomized services and card templates, supporting multi-terminal two-way preview, low-code development, distributed simulation, and distributed commissioning. According to reports, the current atomic service development efficiency has been reduced from 1 month to 15 days, next year's goal is one week, it is expected that by 2023, it only takes 5 days to complete the development.
Download address: https://developer.harmonyos.com/cn/develop/deveco-studio/index.html
Wang Chenglu said that in the future, we will continue to enhance our capabilities and tools, leaving more time and convenience for developers to innovate in the business, "leaving the complexity to us and leaving the simplicity to the developers." ”
On September 10, 2020, Hongmeng operating system was officially released, taking the most critical step in Hongmeng ecology. On June 2 this year, the release of the harmonyos 2 mobile version pressed the fast-forward button on the Hongmeng ecological construction. Today, there are more than 150 million harmonyos-enabled devices. This achievement can be regarded as a good start for Hongmeng Ecology.
The advent of the true Internet of Everything era is inseparable from AI, which will redefine the way people and devices, people and services interact. Now, Hongmeng has done some work on AI development capabilities and services, but it is not enough to rely on Huawei alone. At the meeting, Wang Chenglu also said that more partners and developers are welcome to join the Hongmeng ecosystem, continue to add bricks and tiles, and open a new era of intelligent connection of all things, which is also the mission of hongmeng ecology since its birth.