Author | Mengyidan Interviewed Guest | Huang Jia
出品 | CSDN（ID：CSDNnews）
Large models are refreshing everything, and software development is moving from the digital era to the intelligent era. Software engineers are standing at a turning point in history, catalyzed by large models, the threshold for programming is lowered, everyone can become a developer, application development is within reach, and programmers are entering a new era that is both full of challenges and infinite possibilities.
The protagonist of this article, Huang Jia, a big data and AI expert who has been working in the software development industry for more than 20 years, was just when he thought he could rely on SAP to eat for the rest of his life, when a quote from a friend who worked as a project manager at IBM opened the door to a new world.
Huang Jia, Chief AI Researcher of the Singapore Research Agency, former senior consultant of Accenture Singapore, has been in the industry for more than 20 years, participated in large-scale projects in various fields such as government departments, banks, e-commerce, energy, etc., and has accumulated rich practical experience in artificial intelligence and big data projects. In recent years, his main research interests are NLP pre-trained large model applications, FinTech applications, and continuous learning.
He has published many best-selling books such as "GPT Illustration: How to Build a Large Model", "Ten Words of Data Analysis", "Learning Machine Learning from Zero", "SAP Programming", etc., and recently published the book "Large Model Application Development: Hands-on AI Agent".
From SAP to big models, 20 years of programmers' transformation
From SAP to the research and development of large AI models, his experience is a microcosm of many programmers and the changing times. In the process, he has written a number of best-selling books on AI, among which "GPT Illustration: How Large Models Are Built" won the Top 10 IT Technology Books of 2023, and readers rated it as "seeing the trees and seeing the forest". Subsequently, he transformed the underlying principle knowledge into specific practical applications, which will be revealed in "Principles and Application Development of Large Language Models". Q: Please tell us about yourself, what you are currently engaged in, the technology stack you use and the technology trends you pay attention to on a daily basis? Huang Jia: I work at the Agency for Science, Technology and Research in Singapore on the research and application of large models. I have been engaged in the design, secondary development and implementation of SAP systems for a long time, and in recent years, I have devoted myself to the training, fine-tuning and application development of large language models. The explosion of large language models is only a matter of the past two years, but in the past few years, our institute has invested a lot of resources in the scientific research and application of language models based on Transformer architecture, such as BERT, T5 and other older generation models. After the advent of the ChatGPT series of closed-source models and the Llama3 series of open-source models, our enterprise cooperation and landing application projects have increased all of a sudden. Many organizations have begun to embed the capabilities of open source or closed-source models into every aspect of their work and products. This kind of application from scientific research projects to enterprises is exactly what we are happy to see.
Therefore, the technology stack that I care about and apply is very simple, which is the training, fine-tuning, and quantization of open-source and closed-source large language models, as well as the application of the capabilities based on these models.
Q: When did you first get involved with programming? Can you share your coding journey?
Huang Jia: In 1988, when I was 9 years old, my primary school purchased a batch of Apple computers as a pilot school. One day, my homeroom teacher suddenly asked me: Do you know 26 letters of the English alphabet? At that time, my brother used to teach me some English, but I didn't know the teacher's intentions and misunderstood her questions. I thought it would ask me if I could count in English, and I said I could count 20 and count from one to twenty. The teacher said, "Ah, that's fine." 6 more is enough. (The tone of the following speech is very mysterious, as if there is a treasure buried somewhere in the school...... I can't let other students know, I should be afraid of being jealous...... I want you to join our school's computer team, don't tell anyone, we have a few Macs in our school. And just like that, I got into the computer group by accident. I still remember that the display was very beautiful at that time, the white and translucent plastic shell on the back of the head, which was better than the general black and white or color TV at that time, and the screen was green and green, revealing a hint of mystery.
It is this !!! stirred up memories to kill !! The specific learning process is still quite impressive, and it may be that for the 4th grade elementary school students at that time, English was a little interesting. I remember the Go To statement that the teacher talked about one day, this statement is rarely used now haha, it belongs to the kind of programming paradigm that has been abandoned, after all, modern programs are not easy to go to other places casually when running. The teacher writes on the blackboard: Today we are learning the Go To statement! Hahaha, no way, because after all, the teachers and classmates were more accustomed to Chinese at that time. Then, I glanced at the notes of the classmate next to me and saw that what she recorded was: Rabbit fishing statement. Hahahaha, laughed for a day. After studying for more than two years, when I was in the 6th grade, I represented the school in a programming competition, and I remember that the big problem was to write algorithms in the Basic language and print Yang Hui triangles. Because I went to junior high school, I left school, and I didn't know how many prizes I won, and then my younger brother was promoted to the same junior high school, and I asked him if the school said that we won several prizes. He said, "I have won the award, and I have also given the award, and I have received it on your behalf." Q: Have you ever encountered a bug in your career that gave you a headache or even tormented you for days, and how did you solve it in the end? Huang Jia: After being a programmer for so many years, there are not too many moments like this. That's part of a programmer's life, isn't it?
I used to support the Payroll and the National Provident Fund system for civil servants in the country where I was working at the time, doing batch processing of millions of people every night and then sending the documents to dozens or hundreds of departments (various banks, financial institutions). If the system batch job fails suddenly at night, and you get up immediately at 3 o'clock in the morning to debug, find a solution, solve the problem, and if you can't solve the problem, then you should also determine the next plan, such as asking the customer to notify the major banks to postpone pulling the file for 4 hours this morning (that is, the problem must be solved in the next 4 hours, because you can't let the customer repeatedly notify the major banks to postpone the payment). Sometimes, thinking that a solution has been found, a change goes into the production system and messes up other related modules. Strangely, I don't really remember these specific hardships and pains, as if a program or project has passed and is lost in the long river of history, never to be recalled. I think it's all about business-specific programming, and it's not much about system architecture, so the problems I've encountered are very specific, and the solutions aren't generic. However, one thing is certain, and that is:1. All problems are ultimately solved; 2. New questions keep coming back. As a programmer, you are always trembling, always on the edge of the abyss, always on thin ice. Q: In your more than 20 years of technical programming, how have you stayed sensitive and enthusiastic about new technologies? Huang Jia: Stay hungry stay foolish. I think it's a very fortunate thing that life has changed. Actually, I've not always been sensitive to new technologies, but for 15 years or so, I was doing the same thing, which was SAP programming. I kept repeating myself, and then I was complacent, saying that if you look at a technology stack, it can actually be prosperous for more than ten years, maybe it will continue to be popular, maybe I can rely on the exact same technology to eat for a lifetime. It's like the older generation of COBOL programmers are still in vogue, aren't they? Until one day, I finally reached a state where there were no challenges, no new ideas, and no passions. The work is still there, and I'm familiar with it. But I no longer have a sense of meaning in creation. At this time, a friend (senior) who was a project manager at IBM and had also done some coaching in-house told me, "Oh, if you're bored, you can learn machine learning, which is very popular right now, you know?" Have you heard of Ng Ander's course? I also tried to learn, and I still heard clearly in the first two lessons, and I passed, and when I started programming in the third lesson, I couldn't learn. You can try. "Uh-huh. I can try. I guess that's the same. I should give it a try. And so I started. —— The door to a new world has been opened. About 1-2 months later, I got a certificate of completion for Ng's machine learning course on Coursera, and at the same time, my fourth book, Learning Machine Learning from Scratch, was almost finalized (the first three books were all SAP technologies). - Yes, the process of learning and writing is usually the same process for me, because only in this way can I truly record the doubts, worries, mental journey from not knowing to understanding, and all the confusion of beginners. After that, I really fell in love with the field of AI, which was growing very fast, and I felt like I was back as a young man from a semi-retired guy, I went to a research institute, I studied with a group of colleagues, read the latest papers, and did projects. I'm very, very fascinated by the sheer number of ingenious algorithms in the field of AI (for example, the principle of left-handed interaction in generative adversarial networks (GANs), residual links in ResNet, convolutional kernel weight sharing and feature extraction ideas in CNNs, cyclic connections in RNNs, etc.) are too numerous to mention. In short, get back to your passion. Only with passion and change, keeping pace with the times, can you continue to love life.
Huang Jia's latest "Large Language Models - Principles and Application Development Practices" will be launched soon
In the era of large models, which skills will be amplified? Big models are reshaping software development, and Kent Beck, founder of Test-Driven Development, tweeted that while 90% of past skills will be obsolete, the remaining 10% will be magnified 1,000 times. Q: In the "2024 AI Developer Survey Report" released by CSDN, 90% of developers have used code intelligent assistants/generation tools. Jia Huang: These 3 skills may be of great benefit to programmers:
- Deep understanding of the business;
- Strong communication skills, problem-solving is not important, coding is not so important, how to make your leaders understand what you say is still very important;
- Architectural skills and logical thinking – no matter how easy code is to write, it needs someone to tell the AI how and what to design.
Q: If you can speak, you can become a programmer, and in the future, you can program just by learning natural language? Do you think it is still necessary for programmers to learn programming languages in the era of large language models? Or some basic knowledge of basic algorithms and data structures? Huang Jia: The answer is self-explanatory. If your goal is to be an ordinary code farmer, it doesn't matter if you learn it or not, it doesn't matter what you learn. If your goal is to be a skilled craftsman, you use the spirit of craftsmanship to learn. It's like a dish stir-fried by a chef who is proficient in 18 martial arts, and a waiter who cooks pre-made dishes in the microwave is worth the same value to someone who only needs to fill their belly. If our customers are all people who just need to fill their stomachs. In the same way: in the era of AIGC, do we still need artists to paint one stroke at a time?
Q: 2024 is interpreted by the outside world as the first year of large model application, what challenges and problems do you currently face in the application of NLP pre-trained large models? Huang Jia: There are many challenges and problems, and each of them is an opportunity for you and me:
- Unable to actually implement - this is a paradigm change, which is still being understood, this is a paradigm change from rigorous programming logic to natural language programming, and many problems need to be solved;
- Can't find the landing scene - you need a discerning eye to find the scene;
- Security and illusion – As large language models become more and more capable, and the context of RAG becomes longer, the model has a reference, and it is believed that this question will gradually be answered;
- High demand for computing power - Moore's Law will be resolved.
Q: ChatGPT and Llama3 respectively represent the closed-source and open-source models have been implemented in thousands of industries, here I want to insert a controversy that has been accompanied by the technical circle, that is, the open source model and the closed-source model, what do you think of the debate between the two, the future is the large model open source dominate the side or the closed source is far ahead?
Huang Jia: I see it this way, the two will coexist forever and can confirm and promote each other. It's like Unix vs Linux, MS Office vs WPS in history, it's not about who eats whom, but inspiring and stimulating each other's growth.
To be more specific:
- Open-source models, such as Llama3, provide transparency and accessibility, giving researchers and developers the freedom to modify, improve, and adapt these models to meet specific needs. The open source strategy fosters the speed and breadth of innovation, as it allows contributors across the globe to work together to solve problems and share solutions. This model can accelerate the development and adoption of technology, especially in academia and start-ups.
- Closed-source models, such as ChatGPT, are often developed by commercial companies and may have advantages in terms of resources, proprietary datasets, and business-driven innovation. A closed-source strategy helps protect intellectual property and trade secrets, potentially leading to more fine-tuned performance in a particular domain or application.
It's hard to generalize about which model will dominate in the future. The possibility is that open source and closed-source models will continue to coexist, each with its own strengths in different environments and uses. Open source models are likely to continue to gain traction in academic research and open innovation, while closed-source models are likely to dominate in use cases that require a high degree of specialization and commercialization. The choice of model may be based on a combination of factors such as business needs, cost-effectiveness, and technical support. In addition, with the rapid development of the AI field and the gradual improvement of regulations and policies, new models may emerge in the future or adjust existing models. For example, there may be more cooperation models that combine the innovation capabilities of open source and the advantages of closed-source resources to jointly promote the development of large-scale model technology.
This trend could lead to more cross-border collaboration and technology sharing, driving progress across the industry. - As a researcher, as well as as as an application developer, this kind of competition and collaboration is something I would love to see. Because OpenAI and GPT are dominant, it's certainly not good news for everyone.
At present, I am pleased to see that in the field of commercial closed-source models, the Claude3 Opus model has surpassed GPT-4 in many aspects, and of course, GPT-5 will come out again at any time to catch up. In the field of open source models, excellent models such as Lllama3, Mistral series models, and Phi-3 series models are actively approaching GPT-4. We are very positive and look forward to the future of innovation.
Q: The large model agent technology is also very popular, can you share some cases with us, and will you share this content in your course? Jia Huang: Okay, it just so happens that at this point, I'm focusing on the research work of Agent. Agents are automated applications that are driven by large models and make decisions autonomously. The most important thing to learn Agent is two things: one is to learn advanced Agent cognitive frameworks (such as CoT, ReAct, Self-Ask, Plan-n-Execute); The second is to learn development platforms that integrate advanced agent cognitive frameworks (such as LangChain, LlamaIndex, and OpenAI Assistants). From the perspective of the advanced agent cognitive framework, Andrew Ng recently summarized four paradigms.
The four basic framework design patterns are:
- 反思（Reflection）：Agent 通过交互学习和反思来优化决策。
- Tool use: The Agent can invoke a variety of tools to complete the task in this mode.
- Planning: In planning mode, the Agent needs to plan out a series of action steps to achieve the goal.
- Multiagent collaboration: Involves collaboration between multiple Agents. What I just said.. .. are all products of this cognitive model.
Then in the course, I dissected it more specifically, divided into:
- CoT (Chain of Thought) greatly improves the model's ability to deal with complex problems by introducing a coherent thinking process to guide the model to conduct deeper logical reasoning.
- Self-Ask - allows the model to self-generate questions, perform self-queries to obtain more information, and then combine that information to generate a final answer;
- Reflection - increases the ability for self-reflection and self-improvement, not just thinking when solving a single problem, but using long-term and short-term memory to optimize the overall decision-making process;
- Function Callings/Tool Calls - 工具调用机制就如虎添翼 赋予了大模型更多的环境交互和掌控环境的能力；
- The core idea of the ReAct framework is to think, observe and act in repeated cycles, iterations, and constantly optimize solutions, knowing where the problem is finally solved, which not only enables the agent to perform complex internal reasoning, but also reacts and adjusts its behavior in real time to adapt to changing circumstances and needs. This further enhances the interaction ability and application scope of the model, and represents a new milestone in the development of the Agent cognitive framework.
- Plan-and-Solve – Plan first, and take the task step by step, so that you can increase the completion of the task.
We do not just explain these reasoning frameworks theoretically, but all of them are implemented in code, and we learn how to set up the thinking framework of the Agent through practical cases. The next step is to apply the Agent application development framework, or the actual combat of the tool.
For each framework, we will also provide specific business cases for practical exercises.
Q: What programming foundation do you think you need to have to learn large model agent technology? Huang Jia:1. Only a simple programming foundation is required2. Clear Logical Thinking 3. In-depth understanding of the principles of large language models (it is best to tear the Transformer architecture several times by hand)4. Development frameworks/APIs for large language models
In the era of large models, programmers don't need to be anxiousQ: AI technology is advancing rapidly on a daily basis, and every day witnesses the leap forward of technology and the spark of innovation. In the field of language models, Claude 3, GPT-4, and Gemini 1.5 are vying to set new benchmarks, and the open source model community is not far behind, with the active development of projects such as Llama 3, Grok-1.5, and Michtral, how can programmers get rid of the anxiety or anxiety caused by technological change in this era of highly competitive large models? Huang Jia: The only way to learn this is to jump in, and AI technology is not difficult for application developers, and it is a dimensionality reduction blow for developers to do prompt engineering design and large-scale model development. But you need to know what to learn and how to learn. Because the content of this technology stack is very complicated, it is easy to get into a rabbit hole and can't get out, but what you learn is not practical. My advice is to get an overview first, and then find what you like and dig deeper. Finally: don't be anxious and don't feel like you're older (you're probably not as old as me anyway), because it's not just you, it's everyone who is a newbie. Altman was just as shocked as you and me when he first saw GPT-4, he was also a newbie, and everyone, including the creators of GPT-4, understood GPT from 0 just like us. Now GPT is there, waiting for us to start from 0 and think about how to use it creatively. Q: Compared to other large language model courses on the market, can you talk about the highlights of this course? Huang Jia: Let me talk about this course from the following points:
- Practicality, I emphasize code practice. I code a lot by hand, and I try to code with a large model every day to do actual projects;
- It covers a wide range of areas, from the principle and application of large models, to the actual development practice, to the fine-tuning and quantification, to take the overall situation, clarify the context, and the correlation and comparison between various development frameworks;
- Suitable for beginners. My style is from 0 to 1, step by step, beginners usually find it very comfortable;
- Learning in a group, joining a group, and making progress together will make you very rewarding in this learning atmosphere.
Q: In the process of working as a technical coach, what are some memorable students or cases that have you memorized? Jia Huang: My current experience is focused on doing projects, writing books, and columns. My work has given me a whole bunch of supporters and their encouragement keeps me going.
Here are some of the encouragement and comments from some of my readers and fans about my course and work:
- I love the reader's admiration that "finally got rid of the fragmentation and finished the puzzle";
- I like a quote from readers: see the trees, and see the forest. Hopefully, our courses will give learners a holistic view.
There are so many recognitions that I have received, and they motivate me to do my best as a knowledge disseminator and do my part.
For the course outline and more details of "Large Language Models - Principles and Application Development Practice", please scan the course assistant WeChat to receive it.