IDP: Cloud-native "Jupyter" allows algorithm teams to | Q recommendations more efficiently

Author | Wang Yipeng

On January 14, 2022, the domestic self-developed cloud-native AI development and production platform - IDP (Intelligent Development Platform) trial version was officially released. IDP is developed by White Sea Technology, and its goal is to provide easy-to-use AI development and production tools for data scientists, algorithm engineers, etc., to fill the gap of domestic software in this field and create a more extreme experience "Jupyter".

1

The ERA of AI has arrived, but domestic easy-to-use software is still blank

Under the coordinated promotion of the "troika" of data, algorithms and computing power, the application of AI has developed rapidly and has gradually penetrated into all walks of life in the national economy. Along with the large-scale production of AI, the complexity of data, the increasing complexity of models and the exponential growth of computation volume, algorithm development teams urgently need AI basic software such as data governance, model development, and high-performance computing to help them develop and produce AI at low cost and low threshold.

However, at present, the domestic market pays insufficient attention to the pain points of data scientists and algorithm engineers in AI development and production, and lacks easy-to-use, integrated domestic AI development and production tools. Existing market solutions are generally foreign products, which may have slow cross-border use, experience impairment, or high prices, which increase the burden of enterprises using AI technology for intelligent transformation.

Therefore, the founding team of Baihai Technology decided to build an AI intelligent development platform IDP, focusing on empowering data scientists, algorithm engineers and big data engineers who want to transform into data scientists, providing them with easy-to-use AI development and production tools, allowing them to focus more on model development and data analysis itself, improving work efficiency, while helping enterprises improve efficiency and reduce AI development and application costs.

2

Cloud-native, professional lightweight, and on-demand

Enterprise data migration to the cloud has become an irreversible trend, and from the perspective of penetration, cloud services can hardly be called an "emerging technology". According to sequoia China's 2021 Enterprise Digital Annual Guide, more than 70% of enterprises have used cloud services, and 47% are promoting and applying them on a large scale. At the same time, with the complexity of enterprise business and the diversification of cloud service players, multi-cloud deployment has become a new trend.

With the vigorous development of cloud services, cloud-native applications have gradually attracted the attention and love of enterprises with the advantages of rapid deployment, flexible expansion, and cost reduction. As a next-generation AI development and production platform, IDP is committed to building a cloud-native "Jupyter", which combines Jupyter's interactive development experience with cloud-native performance to provide developers and enterprise customers with more easy-to-use, flexible, efficient, and portable AI development and production tools.

IDP is not a tool-integrated, thick, large, and comprehensive machine learning platform, but a professional, lightweight, on-demand tool platform with the IDE as the only entry point.

IDP takes the integrated development environment IDP Studio as the only user interface, through the built-in machine learning libraries / packages that can be flexibly installed and uninstalled, and the functional components involved in algorithm development, etc., so that users can complete the whole process of data preparation, data exploration, model construction, model training, model evaluation model deployment and release in one stop, stable and efficient manner in IDP Studio. IDP Studio also supports data processing using SQL directly, as well as team collaboration programming and real-time sharing.

In addition, it is worth mentioning that in order to meet the performance requirements of professional algorithm development teams, IDP takes the high-performance distributed computing engine IDP Engine as the core support to help enterprises achieve intelligent distributed execution, improve algorithm development efficiency, and reduce resource consumption costs.

3

Ease of use, performance and collaboration

The IDP has three main core characteristics:

Ease of use: Native IDE-level experience

IDP provides the same quality user experience as the local IDE, while meeting the needs of interactive and non-interactive programming, and the programming environment provides code intelligent completion, code automatic repair and other coding aids to improve development efficiency.

Data scientists can work on SQL processing and ML in the same development environment stably and conveniently, without switching tools and platforms, and rely on IDP Studio's built-in mainstream data science library packages and multiple machine learning frameworks to quickly get started with algorithm development.

Performance: high efficiency, low cost

Through visual workflow orchestration and management functions, IDP helps data scientists and algorithm development teams to unify the scheduling of data preprocessing, data exploration, machine learning, deep learning, DB Engine, etc., and improve development efficiency.

IDP can help enterprises carry out refined computing resource management, support preemptive scheduling and automatic expansion of computing resources, and have process-level breakpoint continuous running and automatic sleep functions, helping enterprises improve resource utilization efficiency, reduce resource waste, and shorten training waiting time.

In addition, IDP optimizes the stand-alone version of the code through data parallelism, code sharding, code compilation optimization, etc., to achieve intelligent distributed execution.

Collaboration: Teamwork, multi-cloud compatible

IDP supports collaborative programming, and algorithm teams can maximize team productivity by sharing models and projects in real time, collaborative editing, comment interaction, and more.

As a cloud-native AI development and production platform, IDP naturally supports multi-cloud deployments, providing customers with a variety of convenient choices.

4

Future Focus Plan: Core Open Source

In the next step, White Sea Technology plans to open source the core functions of IDP Studio in IDP. At the same time, Baihai will further improve the product functions, on the one hand, continue to optimize IPD Studio, in order to more efficiently meet the basic needs of enterprises for team collaboration and AI production; on the other hand, consolidate the distributed computing engine IDP Engine, to help enterprises improve the quality and efficiency of algorithm development and training.

Lu Yilei, founder and CEO of White Sea Technology, said: "Improving the productivity of professional data scientists and promoting AI innovation in enterprises is the original intention of IDP, and it is also a beacon to lead IDP's future development. We hope that through the open source IDP Studio, we can make up for the market gap of domestic easy-to-use IDEs, so that the algorithm development team has more professional, simple and easy-to-use tool choices; we hope that through the open ecosystem, we can work with industry partners, developers and customers to promote the development, application and landing of AI, and accelerate the intelligent transformation and innovation of enterprises. ”

Honorary Experience Officer, IDP is waiting for you

IDP Trial Edition is aimed at geek data scientists and enterprises eager to improve the productivity of algorithm teams, open a small number of priority trial places, you will have the opportunity to experience the core functions of IDP first, explore the surprise of domestic AI development and production tools.

At the same time, I hope that you will actively feedback and share the trial experience in actual scenarios, promote the growth of IDP, and create the future of China's AI basic software tools!