Edge AI developers are building AI applications and products for safety-critical and regulated use cases. With NVIDIA Holoscan 1.0, these applications can integrate real-time insights and processing in milliseconds.
With the latest release of NVIDIA® Holoscan 1.0, it's easier than ever for developers to build production-ready applications for multimodal, real-time sensor processing. NVIDIA AI Enterprise now provides security patches, critical vulnerability fixes, and API stability for the Holoscan software stack, reducing product maintenance costs and simplifying the deployment of AI applications at scale.
什么是 NVIDIA Holoscan？
Holoscan, NVIDIA's domain-agnostic, multimodal, real-time AI sensor processing platform, provides the foundation for developers to build end-to-end sensor processing pipelines. Architected with performance, availability, and production readiness in mind, the platform minimizes development complexity while optimizing performance at every stage of the sensor processing workflow. The workflow contains:
- Sensor data entry
- Accelerated computing and AI inference
- Real-time visualization, execution, and data flow egress
Figure 1. The NVIDIA Holoscan platform helps developers transition from fixed-function to software-defined instruments
Specifically, the Holoscan platform consists of three main components. The first component is software, which establishes an all-domain AI real-time sensor processing workflow based on data movement, accelerated computing, real-time visualization, and AI inference.
NVIDIA Holoscan is the only single platform that brings these capabilities together. It not only guarantees application performance, but also reduces complexity and time to market. The NVIDIA Holoscan SDK provides developers with a low-code, high-performance environment that enables them to build workflows in the programming language of their choice via Python or C++ APIs.
The second component of the Holoscan platform is hardware. Holoscan supports a variety of hardware configurations to meet different power, size, cost, compute, and configuration needs, including:
- NVIDIA Jetson Music 系列产品(Nano Music、Music NX 和 AGX Music)
- NVIDIA IGX Orin
- NVIDIA IGX Orin 外加 NVIDIA 独立 GPU,同时具备集成 GPU 和独立 GPU 的功能
- x86 systems (workstation or cloud) with NVIDIA discrete GPUs
The third component of the Holoscan platform is Enterprise Enablement, which is specifically designed to meet the pre- and post-launch needs of medical technology products. Holoscan software services through NVIDIA AI Enterprise provide long-term, 10-year, full-stack software support.
Specifically, the service can release security patches and critical vulnerability fixes to developers, and ensure the stability of APIs, thereby greatly reducing product maintenance costs. This not only simplifies operations, but also gives developers more confidence to deploy AI applications at scale.
Figure 2. NVIDIA Holoscan helps accelerate time-to-market
Production readiness
The introduction of AI computing has opened up a range of possibilities for businesses that are looking for new ways to deliver differentiation and value while making more efficient use of data. However, the AI software stack is complex, with hundreds of influencing factors and the use of a large number of third-party open source software libraries (Figure 3).
An increase in software complexity equates to an increase in vulnerability. Security, stability, reliability, and production readiness are critical, but they can be challenging for developers and businesses looking to leverage this software in their applications, especially for safety-critical and regulated industries such as medical devices and robotics.
Figure 3. NVIDIA Holoscan delivers the production readiness, security, and stability critical to complex AI software stacks
For example, in 2023, the U.S. Food and Drug Administration (FDA) issued new guidance on cybersecurity for medical devices, which states that premarket submissions may be denied acceptance (RTAs) if they do not contain the information required by Section 524B of the FD&C Act (Ensuring the Cybersecurity of Devices). According to the regulations, manufacturers must design and release updates and patches after the product is released, provide a software bill of materials (SBOM) and submit a plan to identify and address "post-market cybersecurity vulnerabilities."
This poses a greater challenge for developers and enterprises in this field to build and maintain AI platforms:
- Maintaining a secure software stack requires a lot of resources.
- Assessment and exploration are required after a CVE scan for further actions and measures.
- Package security updates may require subsequent changes to dependent APIs, which can break backward compatibility.
- The entire stack needs to be continuously monitored and updated.
This means that security patches and vulnerabilities need to be fixed on a regular basis, leaving organizations with two costly options:
- Backport patches and fixes to avoid breaking the stack and triggering potential validation, validation, and recertification requirements.
- Scroll forward, but risk potentially breaking product functionality and adding new bugs, resulting in costly validation, validation, and even recertification cycles.
What production-ready means for Holoscan developers
The Holoscan platform provides an accelerated full-stack infrastructure for processing multimodal data in real time at the edge. Over the past few releases, the features and functionality of the Holoscan platform have focused on high-performance computing, low-latency and high-throughput multimodal sensor streaming, as well as excellent availability and faster time-to-market.
With the addition of NVIDIA AI Enterprise, Holoscan 1.0 increases production readiness, provides developers with end-to-end full product lifecycle support, and accelerates the AI innovation process for enterprises while reducing the total cost of product ownership and maintenance.
Build products on NVIDIA IGX
With full-stack software support, including firmware, operating systems, drivers, AI frameworks, GPU acceleration, and SDKs, developers can get full-stack support from a single source, while ensuring long-term API stability and a unified software bill of materials (SBOM). This service is optimized to meet the needs of edge customers, such as:
- CVE patches and critical vulnerability fixes as well as stability assurance for APIs.
- Flexible software branching with different terms (9 months to 10 years) to meet the needs of developers in different industries and at different stages of product development. The 9-month term support branch is released every 6 months, and the 10-year term support branch is released every 2.5 years.
- Yocto Enterprise Support for embedded developers who prefer to deploy products with custom Yocto-based distributions.
Yocto-based distributions enable minimal SBOMs and require minimal components to get devices up and running. They simplify FMEA, SW/SOUP hazard analysis, cybersecurity hazard analysis, and penetration testing, reducing the regulatory burden and cost of post-market maintenance, and enabling efficient use of embedded hardware resources, including storage, memory, processing power, and more.
For more information on Holoscan Yocto's distributions optimized for edge deployments, see Deploying AI-Accelerated Medical Devices with NVIDIA Clara Holoscan:
Figure 4. Use Holoscan to move applications from development to deployment
Develop, test, or deploy on x86 systems
Developers can use partial stack support on x86 systems (excluding operating systems). For more information, see NVIDIA-Certified Systems. By maintaining consistency across the same branch software stack across x86 and IGX, you minimize the roadblock from development to deployment. To do this, maintain consistency across x86 platforms for development and deployment for testing, validation, and deployment.
summary
NVIDIA Holoscan works with the industrial-grade NVIDIA IGX Orin platform and NVIDIA AI Enterprise to reduce the burden of validating and maintaining edge platforms for enterprises. It enables teams to focus on building AI, co-developing platforms with a growing ecosystem of first- and third-party developers, and rapidly driving business value.
In particular, IGX services for medical-grade production are now available through a number of ODM partners, including but not limited to:
- Advantech MIC-735M-IO
- ADLINK MLB-IGX
- Onyx Healthcare ACCEL-JS2000
- Dedicated Computing M1000
To get started, download Holoscan 1.0 and check out the release notes:
You can ask questions and share information on the NVIDIA Developer Forum:
You can recap NVIDIA GTC 2024 online to learn more about NVIDIA Holoscan and the latest advancements in AI sensor streaming applications: