Matt, an American with a Ph.D. in communications, recently offered a freelance job: becoming part of Scale AI, training AI models at home.
"Doctor of Communication" "Training AI Model", has Matt become a glorious AI programmer? This is not the case. Matt's job is boring: he "picks up" in Scale AI's system, scans through the various replies given to the user by the AI model from the user's perspective, determines whether the AI has said something wrong, and then provides feedback. For example, booking flights for Google's AI training and reviewing which ChatGPT answers will get bad reviews from users.
Scale AI does not produce large models on its own, and it has partnered with many AI manufacturers such as Google's parent company Alphabet, OpenAI, and Meta to provide them with "humans".
It's just that cheap labor from Africa, India, the Philippines and other places is no longer enough to meet the needs of customers. Scale AI has begun to scale back its overseas operations, employing hundreds of thousands of workers in the United States, including intellectuals like Matt.
The 8-year-old large-scale model data annotation company just completed a $1 billion Series F financing in May this year, led by Accel, followed by Amazon, Intel, AMD, Cisco, Meta, Tiger Global Fund and other world-renowned companies. The company expects sales to exceed $1 billion this year, making it one of the top companies in terms of sales among generative AI companies.
And its latest valuation has reached $13.8 billion. That's a good feat for any AI startup, far more than Silicon Valley star Hugging Face's $4.5 billion valuation last August, and closer to Elon Musk's xAI, which was valued at $180 in its latest funding round.
A
Scale AI, which allows humans to do hard work for AI, is already a critical ammunition depot in the current AI race.
When we think of "large model training", we think of using thousands of advanced chips to drive large models and having them analyze tens of billions of bytes of text, but this is actually only the first step - pre-training.
But these alone are not enough to ensure that systems like Anthropic's Claude, OpenAI's ChatGPT, Meta's Llama, and Google's Bard provide the right answers written in a human style.
In order to achieve this, a second step is required: fine-tuning. This involves a lot of manpower, either hired in-house by AI manufacturers or from companies such as Scale, Surge AI, Labelbox, Telus International, etc. These companies provide a large number of people to write the ideal response to the customer's chatbot and teach the bot to provide more "perfect" answers.
Companies that provide data annotation services for AI models are not new, and the last time they took off was autonomous driving.
Scale AI was founded in 2016. In fact, Scale AI has been associated with OpenAI since the beginning, and its incubation in Y Combinator (hereinafter referred to as YC's start-up horse racing program) has been supported by YC even before the project is over. And the president of YC at that time was Sam Altman, who later co-founded OpenAI.
However, at that time, the "thousand-model war" had not yet begun, and Scale AI first caught up with the autonomous driving technology boom that swept Silicon Valley. Autonomous driving required the training of AI algorithms, and at the time, no other outsourcing company had the ability to annotate the 3D images generated by the radar and sensors of autonomous vehicles.
Scale AI's engineers initially spent a few months building a 3D annotation product for Nuro, an autonomous delivery startup. Soon, Alphabet's Waymo and General Motors' Cruise, and even Apple, became customers of Scale AI.
At the end of 2017, Scale AI employed more than 1,000 annotators, mostly in the Philippines. On average, these contractors earn $1.5 an hour and work 10 hours a week.
By 2019, OpenAI had been established for a few years, and its direction was mainly focused on developing large AI models, and then became a customer of Scale AI. However, at that time, large AI model customers were not a key source of revenue for Scale AI.
As the boom in autonomous driving technology gradually recedes and the market returns to sanity, Scale AI has also encountered a crisis. In 2022, Scale AI's revenue growth has fallen by 50%, much to the disappointment of investors.
However, at the end of 2022, OpenAI released ChatGPT, and Scale AI's "second spring" bloomed instantly.
In addition to OpenAI, Scale AI has also reached a partnership with Meta and Google's parent company Alphabet around large models. The company's revenue soared from $227 million in 2022 to $680 million in 2023.
Standing on the cusp, Scale AI shouted the goal of 206% revenue growth in 2024 and hopes to achieve profitability.
B
At this juncture, Scale AI has also begun to make some changes, cheap labor from overseas can only bear very basic tasks, but large model-driven products have begun to "roll" in writing, programming, professional knowledge, etc., and Scale AI needs to upgrade the "mercenaries" in its hands.
In an investor presentation, Scale said it was building a critical AI infrastructure. The company began to establish itself as an "AI data foundry" reminiscent of semiconductor companies.
The founders of Scale AI have also begun to speak openly about the contributions of people with PhDs, or doctors, lawyers, etc., in training AI systems: "We need the best and brightest minds to contribute data. ”
According to a report by Rest of World, Scale AI recently shut down contractor sites in Kenya, Nigeria, and Pakistan. The company's focus shifted to the U.S., recruiting intellectuals to help train the expertise of large models.
About 300,000 people are waiting to be "dispatched" through the work group run by Scale AI's subsidiary, Outlier.
Scale AI "mercenaries" in the United States are not cheap, and the average hourly wage can reach $40. However, this job still can't get rid of the experience of "hard labor".
Melissa Quashie, a freelancer and editor in Massachusetts, is paid $40 an hour at Scale AI. Her tasks include evaluating the different responses generated by the large model, giving scores based on how the model answers questions and the quality of the content of the responses.
For Kwasi, working at Scale AI has been like "the dumbest video game I've ever played." She once spent two hours writing a "three-day recipe" just to improve the chatbot's answers.
In addition, as Scale AI amasses a large number of laborers, supply and demand have begun to become unbalanced. In many cases, the customer tasks assigned by Scale AI are no longer able to meet the needs of "mercenaries". Many people find that the job, while flexible and well-paid, is often left with nothing to do. Most of the 10 Scale AI "mercenaries" interviewed by The Information had the same complaints.
Maybe it's because the company's business is expanding too fast under the wave of AI, or maybe it's because Scale AI is more focused on serving customers than the work experience of labor. In short, Scale AI is also starting to expose other problems, in addition to the lack of work done by the Spitters, people are also complaining about its lack of training and frequent system crashes.
Even more annoying is the salary settlement, even the "knowledgeable people" who provide labor for Scale AI in the United States have little say. Dr. Matt, mentioned at the beginning, said that he was kicked out of the platform by Scale AI for no reason.
The settlement of remuneration does not depend on the workload, but on the quality, and the final interpretation right naturally belongs to Scale AI. And even if you should be paid, you may not be able to get it because the customer has not confirmed it for a long time.
C
Billing coolies based on the quality of the work rather than the amount of work it does, which helps Scale AI control costs, which is a key point for Scale AI at this stage.
As the company shifts its focus from overseas markets that provide cheap labor to the U.S., the cost of Scale AI is also more difficult to control. According to financial data obtained by The Information, Scale AI's gross profit margin, including the cost paid to human laborers, fell from 59% in 2022 to 49% in 2023.
At the same time, Scale AI told investors that it was working to reduce costs. The company forecasts an increase in gross profit margin by 5 percentage points this year and then to 60% by 2025.
The company told investors that it is reducing the cost of training models by using in-house tools to automatically identify "high-performing experts," as well as relying on computer-generated data to increase the efficiency of human work.
Another way to reduce costs is to reduce internal staff (unlike "mercenaries", in this case, regular employees working at Scale AI), in February 2023, Scale AI has seen the coming of the AI wave on the one hand, and the macroeconomic impact on the other hand, and the wave of layoffs in Silicon Valley, it has seized the opportunity and also cut 20% of its workforce.
In addition to trying to reduce costs, Scale AI is also looking for ways to expand its business.
While many employees objected, Scale AI has long since put behind its promise not to work with the government. In recent months, Alexandr Wang, the co-founder of Scale AI, has taken the stage with U.S. Army generals in Washington, D.C., and the company earns more than $100 million a year from government contracts. In addition, he travels to Qatar for closed-door meetings with government officials, who are also keen to develop their own large language models.
In addition to providing a large amount of manpower to AI manufacturers, Scale AI also provides AI-generated synthetic datasets – using AI-generated data to train AI to meet the growing appetite for AI large model training.
At this stage, "high-quality humans" are still the most important "resources" for Scale AI's survival, so the company is also taking measures to maintain those leaders among the "high-quality humans".
In Austin, Texas, and Jacksonville, Florida, Scale AI held a multi-day workshop with dozens of "top coolies" to attend.
A person who attended the Austin workshop said about 50 trainers were involved in a project that was known to be related to Alphabet's Bard chatbot. They discussed the responses each wrote down for different prompts and sang karaoke together in the evening.
In Jacksonville, Kwasi met university professors, doctoral students, screenwriters, and podcast hosts. "We work six hours straight and then have a glass of wine."
"Everybody is very excited to improve the large language model. But what no one talks about is that because we're doing this job, who is going to lose their job? ”
The irony is that hundreds of thousands of humans are working for AI to make it better and better. And when the AI is good enough, these coolies may also be the first group of people to be abandoned. After all, if AI can be produced and sold on its own, why rely on "Kochi coolies" that cost $40 per hour?
Perhaps a further question is how long the days of selling human labor for AI can continue, and this is a sword hanging over Scale AI's head.