Hot of the week
1. Robots "take over" the Asian Games
2. AI computing power has increased by 680 million times in 70 years, and all aspects of AI capabilities will surpass humans in the future
3. The study claims that the consistency between GPT-4 review opinions and human review is more than 50%
4. GPT-4V debut: support picture and voice input
5, Meta released Llama 2 Long, supporting 32,000 tokens
6. Baidu released the first quantum field large model and two AI native applications
Hello everyone and welcome to our AI Week Hot Tweets! Here, we will present you with the most cutting-edge and hot artificial intelligence information, covering machine learning, deep learning, natural language processing and other fields. Our goal is to keep you abreast of the latest developments in the field of artificial intelligence, grasp the development trends of the industry, and provide a useful reference for your career development and scientific and technological innovation. Welcome to like, follow and share, let us work together to explore the infinite possibilities of artificial intelligence!
01
Artificial intelligence industry dynamics
1. Robots "take over" the Asian Games
The 19th Asian Games were held in Hangzhou, and about 12,000 athletes, thousands of media personnel and technical officials gathered in Hangzhou. From autonomous mosquito killers to simulated robotic pianists and driverless ice cream trucks, machines have taken over the world — at least at the Asian Games in China. Hangzhou is a major hub for China's tech industry, where robots and other eye-opening installations will provide services, entertainment and security for visitors.
02
Artificial intelligence research dynamics
1. AI computing power has increased by 680 million times in 70 years
AI computing power has increased by 680 million times in 70 years, and in the future, AI capabilities in all aspects will fully surpass humans. Within 10 years of the birth of electronic computers, the first AI applications in human history appeared. More than 70 years later, AI models can now not only write poems, but also generate images based on text cues, and even help humans discover unknown protein structures. Computing power, available training data, and algorithms are the three main elements of AI advancement. The three eras of AI development are: the era of artificial learning, the era of deep learning, and the era of large-scale AI models. Future advances in AI technology will depend on the growth of computing power and available data. Startups in the AI industry have raised $14 billion, and the generative AI space is gaining momentum.
2. The study claims that the consistency between GPT-4 review opinions and human review is more than 50%
Researchers at Stanford University and other institutions released papers saying that GPT-4's review opinions for nearly 5,000 papers such as Nature and ICLR are more than 50% similar to those of human reviewers, and more than 82.4% of authors said that GPT-4's opinions were quite helpful. James Zou, author of the paper, concludes: "We still need high-quality human feedback, but LLM can help authors improve their first drafts before formal peer review.
3. A number of institutions in China and the United States jointly released a review of large model interpretability technology
A number of Chinese and American institutions (New Jersey Institute of Technology, Johns Hopkins University, Wake Forest University, University of Georgia, Shanghai Jiaotong University, Baidu, etc.) jointly published a review of large model interpretability technology, which comprehensively sorted out the traditional fine-tuning model and the interpretability technology of Prompting-based super-large model, and discussed the evaluation criteria and future research challenges of model interpretation.
4, Stanford and other open source efficient memory management mechanism PagedAttention
Researchers from UC Berkeley, Stanford University, and UC San Diego proposed a new attention algorithm PagedAttention based on the classic virtual memory and paging technology in the operating system, and created an LLM service system vLLM. According to reports, vLLM achieves almost zero waste on the KV cache, and can flexibly share the KV cache "inside the request" and "between requests", further reducing the use of memory. The evaluation results show that vLLM can increase commonly used LLM throughput by a factor of 2-4, comparable to state-of-the-art systems such as FasterTransformer and Orca in latency levels, and more pronounced for longer sequences, larger models, and more complex decoding algorithms.
5. The research team proposes a new method to improve the reasoning ability of open source mathematical models
A team of researchers from the University of Waterloo and The Ohio State University has proposed a new method for fine-tuning mathematical instructions that can enhance the reasoning capabilities of open-source mathematical models. They created a diverse set of hybrid instruction fine-tuning called MathInstruct, covering different mathematical domains and levels of complexity, and combining both methods of thought chain and thought program. By fine-tuning on MathInstruct, they got models of different sizes and found that these new models outperformed previous models on out-of-domain datasets. This research provides new ideas for improving the capabilities of open source mathematical models.
6. Chinese researchers have launched a large-scale real-world multi-view dataset called "FreeMan"
Chinese researchers have launched a new large-scale, multi-view dataset called "FreeMan" that aims to address the limitations of existing datasets when performing 3D human pose estimation in real-world scenarios. The dataset contains 11 million frames of 8,000 sequences, shot using 8 synchronized smartphones in a variety of scenarios. The researchers generated accurate 3D annotations by automating the annotation process, a dataset that is valuable for multiple tasks, including monocular 3D estimation, 2D-to-3D conversion, multi-view 3D estimation, and neural rendering of human subjects. The researchers also demonstrated FreeMan's superior generalization capabilities in real-world scenarios by comparing with existing datasets. This research is expected to advance in the fields of human modeling, computer vision, and human-computer interaction, bridging the gap between controlled laboratory conditions and real-world scenarios.
03
AI Enterprise Dynamics
1. GPT-4V debut: support picture and voice input
OpenAI announced a new version of ChatGPT, adding two new features: voice input and image input. When using the voice input function, the user simply presses a button, says their question, and ChatGPT converts it to text, then generates an answer, and then converts the answer into speech and plays it to the user. When using the image input function, users can take pictures of things that interest them and upload them to ChatGPT. ChatGPT tries to identify what the user wants to ask and give an answer accordingly. Users can also use the in-app drawing tools to help express their problems, or communicate with voice or text input.
Meta released Llama 2 Long, supporting 32,000 tokens
Meta released Llama 2 Long, with a context length of 32,000 tokens, which is on par with GPT-4. It surpasses Llama 2 in performance, surpasses ChatGPT in test sets such as instruction tuning MMLU (5-shot), and outperforms Claude 2 with 100,000 tokens in human evaluation. According to reports, compared with Llama 2, Llama 2 Long's changes mainly have two aspects. First, in terms of training parameters, a data source of up to 400 billion tokens is used, and the most version of Llama 2 is only 70 billion. Second, architecturally, it remains the same as Llama 2, but makes a very small necessary modification to the position encoding to complete the context window support of up to 320 million tokens.
3. Alibaba Cloud Tongyi Qianwen 14 billion parameters Qwen-14B released
Alibaba Cloud held the Tongyi Qianwen Open Source Conference and officially released the 14 billion parameter model Qwen-14B and the dialogue model Qwen-14B-Chat, which are open source and free. According to reports, Qwen-14B stands out among many open source models within 20B of the same size, and has achieved the best results in 12 authoritative evaluation sets such as MMLU, C-Eval, GSM8K, MATH, and GaoKao-Bench, surpassing all SOTA large models in evaluation. In addition, the Tongyi Qianwen team has upgraded the ability of Qwen models to connect with external systems, allowing developers to implement complex plug-in calls through simple operations, or quickly develop AI systems such as agents based on Qwen series base models, and use Qwen's understanding and planning capabilities to complete complex tasks. At the same time, Qwen-7B has also achieved a comprehensive upgrade, with core indicators increased by up to 22.5%.
4. Baichuan Intelligent released Baichuan2-53B closed-source large model
Baichuan Intelligent released the Baichuan2-53B closed-source large model, comprehensively upgrading the capabilities of Baichuan1-53B. According to reports, Baichuan2-53B's mathematical and logical reasoning ability has been significantly improved, and the model hallucination has been greatly reduced through high-quality data system and search enhancement, which is the largest model with the lowest hallucination problem in China. Baichuan Intelligent also opened the Baichuan2-53B API interface this time, announcing its official entry into the To B field.
5. Open source and commercially available Chinese version of Llama 2 released
Now, 15 hours, thousands of dollars, and 8.5 billion tokens of data can train Chinese version of Llama 2. Colossal-LLaMA-2 recently lowered the threshold for large models, and the open source team also provides a complete evaluation framework ColossalEval to achieve low-cost reproducibility. The comprehensive performance of this model reaches the level of the open-source community's pre-trained SOTA model from scratch at the same scale. The solution is completely open source, including a full set of training processes, code and weights; And there are no commercial restrictions, and it can also be transferred to any vertical domain and low-cost construction of pre-training large models from scratch.
6. Hang Seng Electronics Large Model LightGPT opens internal testing
Hang Seng Grand Model products have started internal testing for 20 financial institutions. It is reported that LightGPT, a large model of the Hang Seng financial industry, is a large language model specially created for the financial field, which has a better understanding of finance-related issues by training massive financial data. Photon is an intelligent application service based on LightGPT, which can inject AI capabilities into various business systems of financial institutions, including investment compliance, investment advisory, customer service, operation, investment research and trading.
7. Baidu released the first quantum field big model and two AI native applications
The 2023 Quantum Industry Conference was held in Hefei, Anhui. At the meeting, Duan Runyao, director of the Institute of Hundred Metric Computing, released the first large model in the quantum field, and two AI native applications, the Hundred Metric Assistant and the Quantum Writing Assistant. He also released a white paper on quantum large models, looking forward to the future development trend and technical potential of quantum large models. According to Duan Runyao, the quantum domain big model is a quantum domain large model built on the basis of Wen Xin's words, using high-quality data in the quantum domain for more targeted training and optimization, which can better understand quantum knowledge and perform quantum tasks professionally.
8. China Telecom released the "Qiming" network model
China Telecom Network Large Model Technology Research Forum was held in Beijing. At the forum, China Telecom released the first network model in the field of information and communication "Qiming". According to reports, the model is independently developed by China Telecom and has been maturely applied within the enterprise. China Telecom will jointly tackle key problems in important links such as basic research, computing power environment, and application deployment, provide various forms of MaaS services, focus on promoting the in-depth evolution of basic general general large models to vertical industry large models, and provide technical support for upper-layer industry application development and open source ecosystem.
9. The first independent and controllable power model in the power industry was released
The power industry artificial intelligence innovation platform and independent and controllable power model conference hosted by China Southern Power Grid Corporation was held in Guangzhou. At the meeting, China Southern Power Grid Corporation released its self-developed artificial intelligence innovation platform for the power industry and the first independent and controllable power model in the power industry. According to reports, the artificial intelligence platform released this time not only provides model-as-a-service (MaaS) solutions, but also supports rapid iterative development of models and is open and shared with the whole society.
10. 49 papers of SenseTime were selected for ICCV 2023
ICCV (International Conference on Computer Vision), a top international conference on artificial intelligence, was held in Paris, France. The total number of submissions reached 8,068 papers in this year's ICCV, of which 2,160 were accepted, with an acceptance rate of 26.8%, slightly higher than the 25.9% acceptance rate of the previous ICCV 2021. A total of 49 papers from SenseTime and the joint lab were selected for ICCV 2023, covering many hot topics related to large models and generative AI, such as Wensheng diagrams, 3D digital humans, autonomous driving, object detection, and video segmentation.
11, Tesla Optimus humanoid robot re-evolution: can rely on vision to classify objects autonomously, and can also do yoga
Tesla's official Twitter account has uploaded a new video: This humanoid robot has evolved and can now rely only on vision to classify objects and perform yoga movements. The video shows that the Optimus humanoid robot uses end-to-end neural network control similar to Tesla's self-driving technology FSD 12: video input, control output, and thus control the movement of individual parts and joints. Optimus completes a simple task of sorting objects by color, placing blocks of different colors into boxes of the corresponding colors.