laitimes

NVIDIA unveils Chat with RTX, which allows users to build AI-powered chat apps locally

author:Mr. Jong Hee
NVIDIA unveils Chat with RTX, which allows users to build AI-powered chat apps locally

In recent years, various artificial intelligence projects have been very popular, and more and more friends have joined the development and learning of various artificial intelligence projects, but the threshold for most artificial intelligence projects is very high, and they are currently mainly concentrated in a small number of enterprise-level and studio-level professional users.

For example, the application of various AI-based large language text models and voice models (such as ChatGPT and speech recognition) requires extremely high hardware computing power and massive training content.

NVIDIA unveils Chat with RTX, which allows users to build AI-powered chat apps locally

In addition, even in the past two years, various AI drawing applications, including Midjourney, Stable Diffusion, etc., have high requirements for the performance of the machine (especially the graphics card), and also require a lot of professional knowledge, which is difficult for ordinary people to play.

The following editor will introduce a lightweight, locally running artificial intelligence project that does not require high hardware and professional knowledge to the majority of friends who are interested in learning and researching artificial intelligence projects. It is important to emphasize that this project does not come from an unknown third party, but from NVIDIA, so there are no security issues.

NVIDIA unveils Chat with RTX, which allows users to build AI-powered chat apps locally

This project is Chat with RTX", an intelligent chatbot application project, very similar to openAI's ChatGPT and Baidu's Wenxin Yiyan, its most intuitive and core function is to accurately answer users' questions.

Regardless of whether the AI project is deployed locally or on a remote server cloud, there is no difficulty in asking questions to it, the difficulty lies in how to accurately understand the user's questions in the background, and then accurately answer the user's questions based on the existing knowledge base (equivalent to the trained model), which is the core.

NVIDIA unveils Chat with RTX, which allows users to build AI-powered chat apps locally

After users download and install Chat with RTX, they can submit various files stored on the hard disk as the knowledge base data source, including .txt, .pdf, .doc, .docx, .xml and other formats, and the background will automatically analyze and organize to generate their own content model, which is stored locally and will not be uploaded to any third party.

The user can then ask it a question, and Chat with RTX can answer the user's question based on the previously summarized knowledge base, and will indicate the source and source of the answer, see the two images above. The overall usage and experience is similar to ChatGPT and Wenxin Yiyan, but it always runs locally and does not require an internet connection.

NVIDIA unveils Chat with RTX, which allows users to build AI-powered chat apps locally

It should be emphasized that Chat with RTX is a hardware and software integration solution, and the project has certain requirements for both hardware and software, but in all fairness, these requirements are not high, and the mainstream configuration in the past two years can be met.

The most important thing is the graphics card, Nvidia said that the project is only supported by RTX 30 and RTX 40 series graphics cards, with more than 8GB of video memory and more than 16GB of memory, the operating system recommends Windows 11, but Windows 10 can also support it, and the graphics card driver version must be at least 535.11 or more.

NVIDIA unveils Chat with RTX, which allows users to build AI-powered chat apps locally

The core of the project relies on NVIDIA's two key technologies, TensorRT-LLM and Retrieval Enhanced Generation (RAG), which NVIDIA will continue to upgrade and improve to improve performance.

TensorRT-LLM consists of the TensorRT deep learning compiler, including optimized cores, preprocessing, and multi-GPU multi-node communication technology, which can greatly improve AI performance. Retrieval Enhanced Generation (RAG) is an important complement to LLMs to improve the accuracy and reliability of generative AI models.

NVIDIA unveils Chat with RTX, which allows users to build AI-powered chat apps locally

In general, Chat with RTX is a relatively complete and lightweight artificial intelligence project, which does not require high hardware requirements and does not require very professional knowledge.

At present, NVIDIA's official website has opened the download of the Chat with RTX project, and those who are using RTX 30 and RTX 40 series graphics cards can visit the official website for more details and install and experience.

Read on