Advances in IoT and 5G technology have led to a growing interest in "digital twins" that digitize the real world in real time in order to turn these data into value by storing them in the cloud, which can be used without having to know the device.
A key requirement for building a digital twin is the ability to process large amounts of streaming data at high speed and add or change processing without disrupting service. In response, Fujitsu Labs developed the Dynamic Reconfigurable Asynchronous Consistent Event Processing Architecture (Dracena) as a reactive platform that can be used to quickly and flexibly develop and deploy services by creating cloud-based digital twins of people, things, and contexts. In reality. The platform has the ability to process large amounts of streaming data in real time and make additions or changes to that processing without shutting down the system. This article describes Dracena's architecture, the technologies developed for it, and the applications that use it.
1. Introduction
Advances in IoT and 5G technology in recent years have raised expectations for services that can turn that data into value by leveraging massive streams of data from many different types of devices in factories, social infrastructure, and homes in real time. These services have entered the commercial use, and large amounts of real-world data are collected in the cloud for visualization, analysis and prediction. Hopefully, they'll also move on to the next step, combining data and services from different industries into ways to control the real world in real time.
One technology that will play an important role in enabling such services is digital twin technology [ 1 ] (Figure 1), which transforms the real world into digital data and links it to the digital space. Digital twins use the cloud to collect real-world (people and things) data and provide the results of analyzing that data to services and applications. In addition, these digital twins represent these raw, real-world data in context that makes sense for real business and life. Doing so makes it possible to efficiently develop services and applications without having to think about the specific IoT devices that collect the real data they use.
Figure 1 Digital Twin
Traditional data processing is performed on batches of data stored in a database. Dracena, on the other hand, performs sequential stream data processing on large amounts of data, enabling real-time analysis and prediction in the real world.
Fujitsu Labs has developed the Dynamic Reconfigurable Asynchronous Consistent Event Processing Architecture (Dracena) as a platform that creates cloud-based digital twins of people, things, and contexts and processes large amounts of streaming data in real time. Dracena can be used to develop and deploy services quickly and flexibly, while allowing additions or changes to be made without shutting down the system.
This article describes Dracena's architecture, the technologies developed for it, and the applications that use it.
2. Prior art and related issues
This section describes three problems with past stream data processing techniques.
1) Difficulty maintaining status information
Identifying when an event occurs in the real world requires saving information about the current state so that changes can be detected in real time. For example, using a stream of data generated by vehicles to determine when a road hazard occurs, you need to be able to detect when data collected from many vehicles traveling on the same road changes from its normal state to a state indicating a dangerous state. venture. Unfortunately, past techniques have achieved high speed by storing only a limited amount of state information in memory or not storing it at all (stateless operations). This means that the database needs to be accessed when information about the previous state is required, which inevitably leads to a significant decrease in processing speed.
2) Difficulty in data sharing and interoperability
The past practice was to integrate systems vertically into silos, where they were tightly coupled from devices to services. This makes interoperation with other services difficult. In addition, how the data stream is formatted depends on each service, and in order to use this data for general purposes, it is necessary to convert them to an easy-to-use data format, which requires not only development costs, but also machine resources to convert.
3) You need to stop the service when you make additions or changes
Given the difficulty of predicting what might happen in the real world, real-world services often need to add and change the way they work. However, for services that provide information for applications such as autonomous driving, such interruptions are unacceptable even for a very short period of time. The way in the past by operating two systems and switching between them only increased the cost.
3. Dracena architecture and implementation
Overcoming these problems requires being able to process large amounts of data from the real world in real time, while also combining uninterrupted operations with agile development and augmentation of different types of services that are relevant to people, things, and contexts. This section describes the architecture and implementation of the Dracena platform developed by Fujitsu Labs for this purpose.
3.1 Schema
Dracena is a platform that provides timely service by processing streaming data from millions or tens of millions of real-world devices, such as connected cars or smartphones, in real time to data centers. Figure 2 shows the architecture of Dracena.
Figure 2 Dracena architecture.
Dracena uses large amounts of streaming data from the real world to maintain in-memory state information for "objects" representing people, things, and contexts. The platform also has the ability to attach plug-in programs that can be invoked by changing the state of these in-memory objects. The functionality of these plug-ins includes the ability to update their own state based on program results, as well as links to pass new event data to other objects for data updates. These features provide Dracena with a reactive architecture where processing is asynchronous and event-driven. They also make Dracena the new real-time processing platform for the cloud era, scalable with high throughput and low latency. The two main features of Dracena are as follows.
1) Real-world objects and service objects
To prevent IoT systems from becoming silos, Dracena extracts real-world data in real time in the cloud to provide a flexible digital twin that represents people, things, and contexts, allowing the addition of service-specific data processing operations [2].
It is also capable of detecting when an event occurs in the real world by storing the data received from the device as state information in a real-world object, keeping it in its original form and preprocessing it for service use.
Objects that hold state information about real-world people and things that are more generic rather than service-specific are called "real-world objects" and objects that perform service-related operations are called service objects. Real-world objects can be shared by different departments within and between companies. In addition, the common data processing between services can be made into microservices as service objects, which can be connected and linked together. As a result, efficient development and event processing are achieved.
2) Seamless upgrade without service interruption
Dracena allows data handlers to be dynamically added or changed without shutting down online services and without affecting scalable processing of millions of real-time data. Figure 3 shows a diagram of how to perform a service update without stopping the operation [ 3 ]. First, when a data handler is upgraded or modified, the program's Java class is distributed to the object as a message as an event in the same way, as shown in Figure (1). Next, take advantage of the Java reflection mechanism to dynamically merge the new handlers, as shown in (2). Finally, enable the service update by coordinating the time to switch to the new handler, as shown in (3).
Figure 3 How to perform a service update without stopping operations.
3.2 Implementation
The Dracena implementation involves the best combination of the latest open source software, including the Apache Kafka [ 4 ] and Apache Flink [ 5 ] distributed stream processing engines for distributed message queuing. These are the only implementations of the objects and plug-ins that make up the real-time digital twin. This implementation approach is in line with this international trend in this era of digital transformation that maximizes the use of open source. Dracena optimizes resource usage by analyzing in detail the performance characteristics of the entire system consisting of these different elements to maximize performance.
4. Potential applications
This section describes typical applications using Dracena.
4.1 Mobility
Figure 4 shows Dracena in mobile. The collection and analysis of sensor data from a large number of connected cars through Dracena can provide a next-generation traffic information service that captures real-time information not only about vehicles and drivers, but also about road obstacles and more. A wide range of changing outdoor environments.
An example is to detect frequent sudden braking on a particular road section and use it to warn approaching vehicles. By analyzing image data collected from passing vehicles, it is also possible to quickly identify and resolve the cause of the problem. Another possibility is to accurately predict potential flooding based on windshield wiper usage and weather data, and use this information to provide safe route guidance.
4.2 Logistics
The spread of e-commerce and the globalization of manufacturing are increasing the complexity of logistics from upstream to downstream of the distribution chain. Access to real-time information on a wide range of changing real-world conditions can help improve job safety and efficiency in such industries, including information such as the status of shipments and work progress at all stages of the supply chain. Distribution centers and warehouses along the way.
By collecting location information for delivery trucks and analyzing it in conjunction with other data, such as changing traffic conditions and unloading space at the destination warehouse, logistics plans can be updated and improved in real time. Real-time analysis of the status of warehouse forklifts and workers can also be used to improve safety or increase work efficiency.
Against the backdrop of urbanization and an aging population, the logistics industry has been lagging behind practices such as using robots, self-driving cars or drones in warehouses for last-mile deliveries. This shows how Dracena will become increasingly important in the future as a digital twin platform for revolutionizing the workplace, which involves the way people and systems coexist.
4.3 Flow of People
The popularity of smartphones in recent years has made it easier to track the flow of people. This information can provide the foundation for applications, such as reducing congestion in stadiums or other event venues by tracking large flows of people and directing them to less crowded routes. Another potential application for tourist attractions is to determine where people are currently, where they come from, and where they stop en route, and to recommend other places to visit.
One consideration in applications that handle people flow is the difficulty of making recommendations that are appropriate for a particular individual. Unlike mobile and logistics, advice that gives people options to maximize convenience may not be what they want.
This is where the digital twin's ability to provide real-world feedback comes into play. Individual characteristics can be understood by tracking how they respond to feedback, and algorithms [ 6 ] can be developed and implemented to provide recommendations that are more appropriate for them.
5. Future challenges and opportunities
Fujitsu Laboratories intends to continue with technology development aimed at overcoming the following two challenges.
1) Provide an efficient development environment
To improve the efficiency of the development of services that interact with the real world, Fujitsu Labs plans to provide a framework that developers can use for simple GUI-based designs and the development of real-world and service objects.
2) Get real-world models quickly
Building digital twins requires models that use the collected data to detect the state of people, things, and contexts, as well as changes in state. Most of the current AI and machine learning techniques for model acquisition work through batch processing, which brings the risk that once a model is acquired, it will soon be left behind by a rapidly changing external environment. Therefore, Fujitsu Laboratories is developing technology to build dynamic models from real-world streaming data in order to gradually keep the models in sync with what is actually happening.
Sixth, the conclusion
This article describes the Dracena Reactive Platform for processing large amounts of real-world streaming data in real time. Dracena allows for flexible and timely service development and enhancements by allowing for additions and changes to streaming data processing without temporarily shutting down the entire service.
In the future, Fujitsu Lab intends to expand the use of Dracena while also focusing on next-generation digital twins that combine features such as automatic learning of real-world models.
Original link: http://www.bimant.com/blog/dracena-digital-twin-platform/