Recently, the Kangaroo Cloud Spring Conference with the theme of "Data+AI, Building New Quality Productivity" came to a successful conclusion, bringing a series of "+AI" digital products and the latest industry precipitation, aiming to closely integrate data and AI, break the traditional productivity boundary, and empower enterprises to achieve higher quality and higher efficiency digital development. At the meeting, the person in charge of the Kangaroo Cloud Data Stack Product Steal Tian brought a new release of the Data Stack V6.2 that integrates AI capabilities, which is not only a simple product upgrade, but also represents a bold prediction of the future of Kangaroo Cloud. (Understanding Kangaroo Cloud https://www.dtstack.com/?src=meiti)
1. Datastack V6.2: Maximizing the value of data
In the data-driven era, data has become the lifeblood of enterprises. How to effectively manage and utilize this data is a question that every enterprise is exploring. The release of Datastack V6.2 is designed to solve this challenge and help enterprises ride the waves in the ocean of data.
The newly released Datastack V6.2 not only provides the basic functions of the big data platform, but also provides enterprises with intelligent data analysis and application through deep integration with AI technology. This means that enterprises can use the data stack platform to achieve the integration of industry content systems, flexible and convenient data insights, the calculation of ultra-fast analysis engines, and the all-round control of data security. In addition, the kangaroo cloud product solution also covers many aspects such as lightweight, data governance, and information innovation. All of this is designed to help enterprises optimize computing and storage costs, improve data quality, promote standards and specifications, and ultimately maximize the value of data.
1. Lightweight data middle platform solution, more efficient data processing
With the introduction of efficient computing engines Doris and StarRocks, a revolutionary refactoring of the platform's performance has been achieved. This innovation not only greatly improves data processing speed, reduces storage and operation and maintenance costs, but also optimizes query efficiency, bringing an unprecedented data manipulation experience to enterprises.
Doris and StarRocks' ad-hoc query capabilities and high-performance analytics and processing capabilities combine to build a powerful and flexible data processing platform. Users can easily cope with the real-time analysis needs of massive data, and realize real-time insight and decision support of data. In this process, the accuracy and reliability of the data are fully guaranteed, providing solid data support for the company's key businesses such as fault prediction, precision marketing and process optimization.
2. All-round data governance system to maximize enterprise value
In Datastack V6.2, we have comprehensively upgraded and redefined data governance to meet the growing needs of enterprises in data management. The five dimensions of the data governance center: storage, computing, quality, specification, and value, constitute a comprehensive data governance system to ensure the integrity, accuracy, and availability of enterprise data.
The Governance Workbench provides an intuitive interface that makes initiating, documenting, assigning, and processing data governance tasks simple and efficient. Through this platform, enterprises can display data governance from an individual perspective, a project perspective, and a panoramic perspective, so as to ensure that the data quality of each link is effectively monitored and managed. The code inspection feature uses SQL check rules to standardize SQL code and prevent possible governance problems in advance. Small file governance optimizes the performance and scalability of the cluster and improves data processing efficiency through one-time or periodic merging of small files in Hadoop clusters.
Data governance in Datastack V6.2 is not only a technical upgrade, but also a shaping of enterprise data culture. Through such a governance system, enterprises can establish a complete data governance framework, promote data standardization and normalization, and ultimately maximize the value of data assets.
3. Full-link information innovation adaptation, supporting comprehensive localization
In this era of equal emphasis on informatization and information innovation, we are well aware of the needs of enterprises for data security and independent control. Therefore, our platform not only achieves full-link information innovation coverage in servers, operating systems, chips, middleware, metadatabases, computing engines, etc., but also makes in-depth adaptation in privatization deployment and whole-process security protection. This is our firm support for the national security strategy and our active fulfillment of corporate social responsibility.
4. Innovate and break through the capabilities of Paimon's data lake to realize the data processing mode of batch streaming
In the traditional data processing model, enterprises are often faced with the dilemma of developing and maintaining two sets of code logic: one for batch processing and one for real-time stream processing. Not only does this mean doubling the development and maintenance effort, but it also requires dealing with the data merging logic between the two to ensure that both systems go live simultaneously. Such a model not only increases the consumption of resources, but also may lead to the problem of ambiguity of data, making it difficult to ensure the accuracy of data and reducing the trust of business personnel in data results.
The data stack innovation breakthrough uses the capabilities of Paimon's data lake to realize the data processing mode of batch and streaming, which effectively solves the above problems. The platform provides real-time lake table development and ad-hoc query capabilities, allowing data developers to process both real-time and batch data on a single platform without additional resources and complex data synchronization processes. Such an all-in-one solution not only reduces the occupation of computing and storage resources, but also ensures the consistency and accuracy of data, thereby improving the recognition of data analysis results by business personnel. This innovation will provide strong support for the digital transformation and intelligent upgrading of enterprises.
5. The four functions of EasyMR are deeply optimized, unlocking a new big data processing and computing experience
As an important product module in the data stack, EasyMR represents our in-depth understanding and continuous innovation of the big data ecosystem. It is based on open source Hadoop and iterated synchronously with the open source community, independently developed by our computing engine team, and optimized and enhanced the features of core components such as Spark, Flink, and Paimon. These optimizations not only improve the performance and stability of data processing, but also give back to the community and promote the co-construction of the Hadoop ecosystem.
EasyMR's capabilities are improved in several ways: it supports hot updates of Flink tasks, ensuring business continuity and flexibility; Spark's Z-Order index optimization and materialized view support improve the efficiency and response speed of data processing. On the other hand, Flink's session class loading isolation ensures the security and reliability of the runtime environment. In addition, the automatic migration function of EasyMR makes it easy to migrate large-scale data clusters, and the status of the migration process is monitored in real time to ensure the security and reliability of data. Through these innovations and optimizations, EasyMR provides users with an efficient, intelligent, and easy-to-maintain big data platform, helping enterprises achieve a qualitative leap in data management and analysis.
2. Data+AI capabilities make data development more intelligent
AI technology has become the core driving force for enterprise innovation and efficiency improvement. By integrating generative AI technology, Datastack V6.2 has implemented six major functions, including intelligent development, intelligent tuning, intelligent diagnosis, intelligent retrieval, intelligent analysis, and intelligent verification, greatly improving the efficiency and quality of data processing.
Intelligent tuning can automatically optimize SQL code to improve execution performance. Intelligent diagnosis uses AI to parse logs, quickly locate problems, and provide professional optimization suggestions. Intelligent analytics help you gain insight into data trends and support decision-making. These features not only improve development efficiency, but also ensure code quality and enable more precise business goals to be achieved in a data-driven way. The introduction of AI+ marks that we are entering a new era of more intelligent and efficient data management.
The AI + intelligent tuning feature can provide intelligent optimization suggestions when developers write code in the editor, so that data development students can review and compare. In order to improve the coding efficiency and code quality, data development students can focus more on the implementation of business logic.
The AI + intelligent diagnosis function uses AI technology to intelligently parse task logs such as Spark SQL and Flink SQL, identify error messages, and provide professional SQL optimization suggestions to help quickly locate the root cause of problems and improve code development efficiency.
Through the integration with AI+, Datastack not only simplifies the data development process, but also improves the accuracy and reliability of data processing, providing solid technical support for data-driven decision-making of enterprises.
3. Product + service, the commercialization strategy of data stack products has been upgraded
In this product launch, we redefined the product commercialization strategy, aiming to provide flexible and diverse service solutions for enterprises with different needs.
The product series includes Standard, Professional, and Ultimate editions, and provides application cloud deployment options to meet the data processing needs of enterprises of different sizes. In addition, we also provide value-added services such as Xinchuang adaptation and real-time lakehouse, as well as the advanced and top-level versions of systematic operation and maintenance services to ensure that customers can enjoy all-round support from basic to advanced. The commercialization strategy of Datastack products not only focuses on the sale of products, but also pays more attention to the continuous optimization and upgrading of services. By providing two paths, product upgrade and version upgrade, it helps enterprises ensure that the data platform is continuously adaptable and future-proof. Such a strategy not only enhances the customer experience, but also lays a solid foundation for the long-term development of the data stack product.
Fourth, three major product practice cases to help the digital transformation of enterprises
1. A bank: AI implementation of performance appraisal
Based on the precipitated performance appraisal indicators, combined with the enterprise's own knowledge base, the bank uses AI intelligent analysis and data processing capabilities to significantly improve the management efficiency and governance level of performance appraisal.
Our solution has helped banks realize the transformation from indicator reports to indicator dashboards and then to indicator conversational BI, greatly reducing the cost of employees taking and using data, making the assessment standards more scientific and rigorous, and the assessment content more complete, ensuring the close connection between the overall performance of the bank and the individual performance of employees. Through AI intelligent attribution and intelligent recommendations, banks are able to track employee performance results in real time, identify problems in a timely manner, and make adjustments, thereby promoting the alignment of employees with the organization's goals and continuous improvement of performance. This shift has not only optimized the bank's human resource management, but has also led to greater operational efficiency and business outcomes across the organization.
2. A Chinese liquor brand: lightweight data center
Through the data stack, the brand has established a unified marketing platform, which helps enterprises realize multi-dimensional analysis capabilities such as data sharing, intelligent labels, and indicator management, and provides strong data support for enterprises' precision marketing and process optimization.
By adopting a lightweight data middle platform solution combined with StarRocks' high-performance computing capabilities, the platform enables liquor companies to achieve efficient data management and real-time analysis. StarRocks' low-latency query and fast data loading capabilities enable enterprises to quickly respond to market changes and achieve fault prediction and precision marketing. Compared with the traditional Hadoop ecosystem, such a lightweight data middle platform solution has the characteristics of excellent query performance, real-time data processing, high concurrency, and easy maintenance in scenarios with small data volumes, making it an ideal choice for rapid data analysis and promoting the pace of digital transformation of liquor enterprises.
3. Beijing Municipal State-owned Group Company: Full-link Information Innovation
In order to solve the problems of enterprise digital transformation and information innovation requirements, this customer has established a "full-link information innovation big data platform".
The platform is deeply adapted to the information innovation ecosystem, and realizes the full-process security protection and privatization deployment from servers, operating systems, chips, application metadatabases, middleware and computing engines. Through such full-link information innovation adaptation, the group not only solves the problem of data islands, but also meets the strict requirements of the state for information innovation and ensures the security and controllability of data. This initiative has significantly improved the Group's data governance capabilities, laid a solid data foundation for the long-term development of the company, and also provided valuable practical experience in information innovation for other state-owned enterprises.
The above is the release introduction of Datastack V6.2, which is not only a product, but also a summary of our deep understanding and practice of big data governance and intelligent analysis. We believe that DataStack V6.2 can help more enterprises maximize the value of data and promote their digital transformation.