How the ERA OF AI expresses the "other world" – the digital twin.
The 2021 Unreal Engine Technology Open House hosted by Epic Games on December 3 ended in Shanghai, and as the annual developer "Pilgrimage" day, the scene was full of people. On the day of the event, nearly 50 celebrities from many industries such as games, film and television, and automobiles were gathered, who shared their experiences from different dimensions such as underlying technical logic, practical application cases, and commercial IP creation.
Among them, Shang Jian, Deputy Director of Tencent Map Technology, shared the keynote speech of Tencent WeMap UE Map Practice, a global map construction scheme based on Unreal Engine, and demonstrated from four aspects: global map, automated modeling, visual polishing, and open system construction.
Based on the exploration and curiosity of digital twin technology, VR Gyro also had an in-depth exchange with Mr. Shang Jian after the meeting.
Source: Unreal Engine
The following is a transcript of the speech, with slight modifications:
Global map, 1:1 restore
We should see a lot about the application of digital twins or smart cities, and we, as a mapping team, have our own different starting points when thinking about creating such a visual product. Our starting point is to unlock the value of data and empower industries. In addition, we want to reduce the cost of visual builds.
We disassemble the global map into a series of different perspectives such as global, provincial, municipal, district, park, and urban close-up perspectives, with all elements. In addition, it can be infinitely zoomed to different levels and roam to various areas at will, which is our definition of a global map.
Let's split the concept of a global map. Technical disassembly can be decomposed into two-dimensional maps, we will present vector maps, and three-dimensional maps will carry out key three-dimensional expressions. The construction of the global map will be presented in a sphere manner, first of all, to build a coordinate system, to create a 1:1 restoration of the real earth, and finally through the conversion of the geographical coordinate system and other technologies, to achieve 1:1 restoration of the real earth rendering.
In the process, we also experienced a series of explorations, during which we also encountered many technical difficulties. For example, the same height may be the presence of a large number of green ground or background surfaces, which are prone to flicker problems, and the distance between different layers and Camera can be pushed away by pixel depth shifting. Due to the large number of layers, we have also done more optimization of the algorithm, added different calculation strategies based on the Camera perspective transformation, and formulated better schemes based on UE characteristics.
After solving these problems, the two-dimensional vector ball has been more completely presented, and you can roam to any area and see the effect of the white mold pulling up. For the three-dimensional part, we abstract to several key elements, such as buildings, roads, vegetation and terrain and other elements, the building can be split into several L1 ~ L5 levels, namely L1 white mode, L2 texture map, L3 structured presentation, L4, L5 artificial model, we will focus on the construction of L3 to do the presentation of three-dimensional maps today.
3D automatic modeling, data lightweight technology
We have global compliant map data, and based on this, we are also thinking about whether there is a more cost-effective automated modeling solution. Take the effect in the video as an example, the urban white mold itself is based on the global map data generation, and a small number of fine molds are added to express the landmark buildings of the city.
Taking architecture as an example, based on our data, we can know the bottom contour of the building as a whole. At the same time, we also know what type of building this is, such as a hospital or a school. Based on the algorithm of image recognition, you can also know the structure and color of its roof, the material and color of the side, etc., and the architectural style is hutong, tube building and so on. Based on this, a rich library of materials has been built.
With the expression of architecture, the next step is to look at how roads and vegetation are constructed, they are very critical elements, and we have data on roads across the globe. Based on more rule additions, in the process of modeling overpasses, intersections, viaducts, etc., automated generation will also be used to finally complete such a differentiated model construction.
With buildings, vegetation, roads, and automated construction solutions, global 3D models are basically ready to be built. The next step is to think about how unreal engine will finally render, but until then it will need to integrate different kinds of data, automated modeling, artificial models, and external multi-source data through WeMap Base's data fusion capabilities.
Of course, such a large amount of data directly on the Unreal Engine is still more difficult, we will use automated modeling in the three-dimensional data cutting technology, whether it is roads, vegetation will enter the three-dimensional pipeline, a series of three-dimensional lightweight work, will export the data of the custom protocol. Finally, after the service automation and cluster deployment, it is pushed to Unreal Engine for the final rendering.
Template design, custom visual effects are diverse
Combined with Unreal Engine's powerful rendering capabilities, we built cool earth effects, architectural effects, day and night changes, and weather systems to enhance the overall rendering visual effects. Such as the presentation of the earth, in order to express more realistic, we increased the presentation of the satellite map texture, in order to make it have a sense of bump, we combined with the terrain to add a normal map, through the reflection of the water surface to make the ocean more textured, through lighting, atmospheric fog and cloud environment, so that the environment is more textured.
In addition, we have customized the entire material system. Enhance extensibility with a unified design. One of the highlights of WeMap products is the template design, the traditional map provides personalized basemap function, users can freely choose a variety of templates, we are also based on this consideration, to provide flexible selection and configuration of style templates, and to achieve real-time template switching.
Open system, integrated API design
When WeMap has a complete three-dimensional earth form and has been further improved, we will think about how to build an open system and launch this set of products for more customers to use. Plugins are a very good solution, and here combined with the scarcity of market UE research and development, we built a set of open patterns based on Web APIs, through cloud rendering to open up the channel between Unreal Engine and the Web side, based on another Web side visualization engine in the team, we created an integrated API design. Students who have done mini programs may know that mini programs can be developed on the PC side and achieve a consistent experience on Android and ioS, and we are similar to the goal of achieving the design of the entire API.
Of course, we will also consider that the cloud rendering method will be affected by the network or bandwidth, so we have created a set of CS deployment solutions to achieve similar effects developed on the PC side.
The following is Mr. Shang Jian and VR Gyro Q&A session:
First, how does Tencent Maps Industry Edition WeMap realize data collection and processing, and what capabilities are needed to support the process of opening up data and integrating data?
Shang Jian: We are a product of Tencent Maps, so we actually have a channel such as map data, including all the services mentioned earlier, which are actually based on map data. The task of Data Factory Data in 1+1+3 (Data Factory, Data Management Middle Office, Intelligent Analysis Platform, Visualization Platform, and Industry Map Service) is to summarize data.
Specifically, first of all, based on the two-dimensional data that Tencent Maps can summarize, this is the vector data we have been building, and then the data will enter the data management middle platform Base for storage and management. Due to the large amount of data entering, Tencent Maps will build a standard data classification system for data fusion and processing, and then the data will enter the three major service platforms, namely intelligent analysis platform, visualization platform and industrial map service.
Overall, from data collection, processing to application, this is a complete set of industry map system.
Image source: Network
Second, weMap modeling has two kinds of white mold and fine mold, so what is the proportion of these two models in the industry map?
Shang Jian: In fact, we divide urban modeling into L1 to L5, where L4 and L5 are manual models, and L3 and below are automated models. Some landmark buildings in a city are usually finely polished, such as areas such as the Bund in Shanghai, while other areas are generated by automated modeling. There are generally no more than 100 dies used in a city, so they are basically generated automatically.
Third, what kind of effect level can WeMap achieve in modeling at present?
Shang Jian: Today, in the theme sharing, in fact, we only showed one set of effect templates, because the needs and preferences of customers are different, so WeMap has actually built a number of different sets of different effect templates. Based on the templated design, users can constantly switch, and even adjust and improve it.
When building from a two-dimensional map, users can generate new effect templates by changing the relevant parameter configurations, and in the process of building our three-dimensional maps, we also hope to develop in such a custom direction, rather than just providing a fixed template.
The effects that can be configured include elements such as underlying effects, lighting, and weather, and WeMap also hopes that after opening up in the future, more developers can join in to adjust it independently.
4. What can WeMap cover in terms of application scenarios?
Shangjian: WeMap's current business is actually from the perspective of industry solutions, empowering various industries from the underlying logic of 1+1+3, among which the more critical application scenarios include smart cities, smart transportation, emergency scenarios, cultural tourism, retail, real estate and other scenarios.
Based on the capabilities of spatio-temporal big data and visualization, combined with the spatial data and business data of enterprises and governments, it provides comprehensive and powerful analysis capabilities and visual presentation for urban governance, and finally builds the underlying solution of a map.
For traffic scenarios, WeMap builds a full-process solution of real-time situation awareness, historical change traceability, and spatio-temporal simulation deduction of traffic business based on spatio-temporal perception, data management, intelligent analysis and visualization capabilities of people, vehicles, roads, places and things.
What stage of development is WeMap currently in?
Shang Jian: WeMap is not a new product, it was released by Tencent in November last year, and this year it made further upgrades of capabilities at the Tencent Digital Ecology Conference, such as in the upgrade of visualization.
In fact, we belong to Tencent location services, which have been precipitated for many years, such as location positioning applications on WeChat, Meituan takeaway and Jingdong online shopping, etc., many of which are supported by Tencent's location team. Including Tencent Maps, it actually has a history of 10 years, and the foundation is relatively deep. On these basis, we go to release a product that combines the industrial Internet, that is, WeMap.
6. In implementing the weMap underlying build, what unique support will be brought by using Unreal Engine to build?
Shang Jian: Mainly in terms of visual effects, the starting point for our team to use Unreal Engine is that its visual experience is very good, such as atmospheric fog, clouds, lighting, lighting effects, etc., we will make the most of it and make it play its maximum value. In addition, the openness of Unreal Engine is also very important to us.
While using Unreal Engine, WeMap will also make some innovations in it, incorporating its own design ideas and styles.
What is the connection between digital twins and WeMap?
Shang Jian: The digital twin is very compatible with WeMap, first of all, WeMap itself is a product built in the direction of the industrial Internet; secondly, based on the gene of WeMap map, we have our own vision of restoring the real world.
Digital twin is a mapping of the real world to the virtual world, with IOT devices as the carrier, so that more users can see a digital "world", these virtual and real data fusion features, is the core of the digital twin, which coincides with the positioning and application of WeMap.
Of course, WeMap is not only for digital twins, digital twins are not only a map baseboard, but also a variety of capabilities to support, I think the combination of the two can play a greater value.
Eighth, according to our understanding, most of the word twin projects are customized, it is difficult to apply as a large-scale product, do you think that such an application status will limit the development of the underlying technology? Which scenarios can exert commercial value on a large scale in the future?
Shangjian: Too much customization stems from the fact that there are many underlying data, many formats, many types and many channels, and many projects are spliced with various different data. Including the construction of three-dimensional data is also a variety of states, and we can also see a variety of problems in this industry.
In this regard, WeMap is moving in the direction of diversification, openness, standardization and low cost of global maps, simplifying complex problems and thinking about different market needs from the perspective of customers. In the open WeMap system, users can build a personalized business system on top of the basemap, including adding annotations, drawing lines, making heat maps, migration maps and a series of other things, so that users can play freely.
In addition to the application of the B-side, Mr. Shang Jian also stressed that in the future, it may be explored in the field of VR/AR, and weMap is mainly in the technology to continue to deepen. As an application that positions the Internet industry, Tencent WeMap is building a visual 3D open map system, based on its open core layout, I believe that in the near future, we will see it burst more vitality on the B-side and C-end.