AI seems to be permeating every corner of modern life, from music to media to business/productivity to private dating. To keep up in this fast-moving era, it's important for everyone to take time to get to know everything about AI.
Artificial intelligence, also known as machine learning, is a software system based on neural networks that pioneered decades ago. With the recent rapid development of new computing power, AI has finally gained efficient and reliable speech and image recognition capabilities, and even mastered the know-how to generate images and speech. Researchers are now working on AI to help users easily summarize web content, order items, and adjust recipes.
Will the machine explode and quickly break out of human control?! Don't worry, this issue will be discussed seriously later. In contrast, we hope that after reading this article, everyone can grasp the current development trend of AI.
AI basics
The most interesting fact about AI is that although its core concept was born more than 50 years ago, not many technologists are really familiar with the principles until now. So if you're feeling lost, don't worry – everyone else is just as well.
Here we want to emphasize one point first: although the name is "artificial intelligence", the word itself is not accurate. There is no uniform definition of intelligence, and AI systems behave more closely to computers than to human brains. It's just that the input and output of this computer are more flexible and can mimic the performance of intelligence to a certain extent.
Here's a look at the basic terminology often used in AI discussions.
Neural network
The human brain is mainly made up of interconnected cells called "neurons" that mesh with each other to form complex networks that can perform tasks and store information. Since the 1960s, people have been hoping to rebuild this amazing cognitive system in software, but it is not until the last 15 to 20 years that the development of GPUs has allowed number-defined neural networks to flourish, which can be described as a typical case of computing power miracles.
Essentially, a neural network is a combination of a large number of points and lines: points represent data, and lines are statistical relationships between values.
Like the human brain, this basic principle enables the creation of a multifunctional system: it quickly receives input, passes it through the network, and generates output. Such a system is called a "model".
Model
A model is a collection of concrete codes that can receive input and return output. The word "model" was chosen to reflect similarities with statistical models or modeled systems that can simulate complex natural processes. In the field of AI, a model can refer to a complete system such as ChatGPT, or almost any AI or machine learning structure with unlimited purpose and function. The size of the model varies, and its size represents the amount of storage space occupied and the amount of computing power required to run. The actual volume is determined by the training method of the model.
Training
To create an AI model, you first "feed" the neural network that forms the basis of the system with the vast amount of information carried by the dataset or corpus. In doing so, a large network creates a statistical representation of that data. The training process is also the most computationally intensive and often requires weeks or even months to run on large-scale, high-performance computers. This is not only because the networks themselves are complex, but also because data sets are often extremely large: billions of words or images must be analyzed and represented in huge statistical models. But after the model is trained, researchers can find ways to "slim down" it, and the resource requirements at runtime are lower — this is called the inference process.
Inference
Inference is the process by which the model actually works: taking the lead in reasoning about existing evidence to draw conclusions. Of course, this is different from our human "reasoning", AI models statistically link the ingested data points to predict the location of the next point. For example, suppose it is asked to "complete the following sequence: red, orange, yellow..." It realizes that these words match a certain list ingested, that is, the color distribution of the rainbow, and then infers and completes the rest of the list. Inference typically consumes much less computation than training: after all, querying the catalog is much simpler than organizing it. While some large models still rely on supercomputers and GPUs to perform inference, many small models can already run on smartphones or even lower-speculated devices.
Generative AI
Today, everyone is talking about generative AI. This is a broad term for AI models that are capable of generating raw output, such as images and text. Some models can be summarized, some can be organized, and some can be identified — but at least the hottest players at the moment are those AI models that can generate new content "out of thin air" (whether this is really out of thin air is still debatable). But keep in mind that the results generated by AI are not necessarily correct, or even nonsense! Everything could be neural network whimsy, including colorful stories or lifelike paintings.
AI buzzwords
Speaking of the basics, let's take a look at the more popular AI words in 2023.
Large language model (LLM)
Large language models have become the most influential and widely used form of AI today, and almost all the text and English literary materials that make up the network have been included in the training category. What is trained from this is a huge set of basic models. Large language models can converse and answer questions in natural language, mimicking various styles of written documents, and ChatGPT, Claude, and LLaMa have all proven their power. While these models perform impressively, note that they are still essentially pattern recognition engines—when answering questions, they are actually completing the recognized pattern, but cannot tell whether the pattern matches the facts. LLM often produces "hallucinations" when answering questions, which will be further expanded on later.
Foundation model
Training a giant model from scratch on top of a huge data set is undoubtedly an expensive and complex process, and it should be avoided. The basic model belongs to a large model trained from scratch and needs a supercomputer to be able to bear it; But we can usually reduce the amount of parameters in it to accommodate a smaller load in a streamlined way. The so-called parameters, that is, the number of "points" to be processed in the model we mentioned earlier, the current common large language models often have millions, billions or even trillions of parameters.
Fine tuning
Basic models such as GPT-4 are very clever, but they can only be considered "generalists" in design. From literary masterpieces to fantasy stories, it has everything covered. But if you want it to help put together a resume for a job search credit, its performance is not even as good as that of the average middle school student. Fortunately, we can use a special dataset to do some additional training on the model, which is model fine-tuning. For example, we can collect thousands of job applications from the Internet, and after "feeding", the model finally understands the routine of the resume, without affecting the other knowledge it has mastered in the original training data.
There is also human feedback reinforcement learning (RLHF), a special fine-tuning method that improves the communication skills of models through data from human interaction with LLM.
Diffusion
Image generation can be achieved in a variety of ways, but the most successful method to date is "diffusion" technology. Popular generative AI core achievements such as Stable Diffusion and Midjourney have developed from this. When training a diffusion model by presenting images, the images degrade as digital noise is added until the original image is gone. By observing the entire process, the diffusion model learns how to reverse the entire process, gradually adding details to the pure noise to form a predefined arbitrary image. In fact, we have explored newer and better implementation methods in the field of image generation, but diffusion technology is still relatively reliable and relatively easy to understand, so I believe there will be a lot of application space.
Hallucination
The original concept of "hallucination" refers to situations where the model mixes the output with completely unrelated content to the input. For example, because the training material contains a large number of dog elements, the model will occasionally use dogs as textures to attach to buildings. According to speculation, the illusions generated by AI today mainly stem from the lack of sufficient data in the training set, or the conflicting data content, so it can only make up some specious conclusions.
The existence of "illusions" has advantages and disadvantages: the use of illusions can guide AI to generate original or more diverse derivative art results. But if you need to get a clear answer to the facts, hallucinations are bound to be a big problem — models will talk serious nonsense and mistake users who are not familiar with the facts into believing that they are true. At present, there is no simple way to judge whether the AI output is true or false other than manual checking, after all, the model itself does not have the concept of "true or false" at all, but is trying to complete the "suspected" pattern it recognizes.
Artificial General Intelligence (AGI)
General artificial intelligence, also known as strong artificial intelligence, does not actually have a clear conceptual definition. In the simplest terms, this is an intelligence powerful enough to not only do a lot of work for humans, but even learn and improve itself like humans. There are concerns that this cycle of learning, integrating thinking, and then accelerating learning and growth will perpetuate, eventually creating a superintelligent system that cannot be constrained or controlled. Some even argue that research should be halted to suspend or prevent this dire future.
Friends who have seen "The Matrix" or "Terminator" movies will surely understand the concern, after all, the possibility of AI getting out of control and trying to eliminate or enslave humanity is indeed chilling. But these stories are purely the imagination of the screenwriter and have nothing to do with reality. While impressive with achievements such as ChatGPT, they have little to do with "true intelligence" when it comes to abstract reasoning and dynamic multi-domain activities. We can't yet say how AI will evolve in the future, but for the time being, we might as well understand AGI as interstellar travel – a concept that everyone can grasp and work towards, but the goal itself is still out of reach. This requires huge investment of resources and leaps and bounds in basic science, and it will not happen overnight.
Commentators have also repeatedly stressed that the "alarmist" discussion lacks relevance. After all, the real threat that AI presents today stems from its limitations and "intellectual disability" manifestations. Although no one wants Skynet to come true, if we can't solve the real problem of automation eliminating jobs in the early stage of AI, how can we still have a chance to be chased and killed by the T-1000 all over the streets?
AI main players
OpenAI
To say that the most famous "school" in the field of AI today is undoubtedly led by OpenAI. As the name suggests, OpenAI emphasizes sharing its research results. But after making some gains, OpenAI decided to reorganize into a more traditional for-profit company, opening up access to high-level language models such as ChatGPT through APIs and application software. The company's head is Sam Altman, who, despite making a fortune on technological breakthroughs, has warned of the risks AI can pose. OpenAI is a leader in the field of large language models and is exploring in other directions.
Microsoft
Microsoft has also made a lot of contributions to AI research, but for various reasons has not been able to really translate the experimental results into real products. But its smartest move was to invest early in OpenAI and build a long-term partnership with the latter. Microsoft has now introduced ChatGPT functionality on the Bing search engine. Although Microsoft's AI contribution is relatively limited and difficult to use directly, its research and development strength cannot be underestimated.
Google, which wants to lead the AI technology revolution with the "moonshot plan", somehow failed to harvest the final fruits of victory. But it must be admitted that the invention of Google researchers laid the foundation for today's full-scale explosion of AI, which is the tarnsformer. Today, Google is working to develop its own big language models and other agents. After wasting a lot of time and money pushing AI assistants to no avails over the past decade, Google is catching up. CEO Sundar Pichai has repeatedly said that the company will firmly adhere to the AI-centric development concept in search and productivity.
Anthropic
After OpenAI "betrayed" the open source community, brothers and sisters Dario and Daniela Amodei left and founded Anthropic, hoping to build an open and more ethically responsible AI research organization. With abundant funding, they have developed into a strong competitor to OpenAI, but their Claude model is not yet able to match GPT in popularity and popularity.
Stability
Despite the great controversy, Stability still has its place in the AI wave. They are collecting all kinds of content on the internet and making their generative AI models freely available in the form of open hardware. This is in line with the idea that "information should be free" and casts a moral shadow over the project itself. Many people believe that Stability's work was used to generate pornographic images and use intellectual property without consent.
Elon Musk
Musk has long been outspoken about his concerns about AI getting out of hand. He had supported OpenAI early on, but was unhappy that the company was moving in a direction he didn't support. Although Musk is not an AI technologist, his exaggerated statements and comments have indeed sparked widespread repercussions (he himself signed the "pause on AI research" initiative) and is working to build his own AI research institute.