OpenAI has hired more than a thousand outsourced people to train AI to think step by step like a human. If ChatGPT "returns from learning", I am afraid that the coder is really in danger?!
Recently, it was reported that OpenAI has quietly trained ChatGPT to learn human thought processes, so as to truly master software engineering and completely replace "junior code farmers".
OpenAI recruits an army of outsourcing and teaches AI to learn human thinking
AI that can program, several big Silicon Valley manufacturers are doing.
DeepMind's AlphaCode, which is said to "beat 72% of human programmers," is not yet available; Google's rumored "mystery project" Pitchfork is also still in the making; Microsoft's GitHub Copilot is primarily a code completion tool.
To say that they completely replace human code farmers, they are not qualified.
But if ChatGPT really learns to program with human thinking, these friends/their own products may be beaten.
By all indications, OpenAI seems to be playing the next big game.
In the past six months, OpenAI has recruited about 1,000 outsourced people from regions such as Latin America and Eastern Europe to train their AI code, according to Semafor.
In this news, there are two "Hua Points".
First, why Latin America and Eastern Europe? We all understand that now that the bubble in Silicon Valley has burst, all Internet giants are racking their brains to "reduce costs and increase efficiency", some rely on layoffs, and some go to other countries to find cheap labor.
The second "Hua Point" is that many of these outsourced personnel are not computer science graduates and do not have advanced programming skills. Their role is to write the "automated" code base that OpenAI expects to achieve.
Specifically, 60 percent of them work on "data labeling" — creating large amounts of images, audio clips, and other information that can be used to train AI tools or self-driving cars.
The other 40 percent are actual programmers who are "hand-rubbing" data for OpenAI's models to let AI learn software engineering tasks.
Previously, OpenAI had been training its models with code scraped from GitHub.
This time, OpenAI wants to build a dataset that includes not only code, but also human explanations written in natural language behind it.
Address: https://arxiv.org/abs/2107.03374
Semafor interviewed a South American developer who completed a 5-hour coding test for OpenAI for free.
In this test, he was asked to handle two tasks.
First, he gets a programming problem, and OpenAI asks him to explain in written English how he's going to handle it.
Then, he needs to provide a solution.
If he finds a bug, OpenAI asks him to elaborate on what the problem is and how it should be corrected, rather than simply fixing it.
"They probably want to feed the model with a very specific training data, in which case they need to show how humans think step by step." The developer said.
In the previous ChatGPT, the code written by it was found to be a lot of problems.
The reason is that ChatGPT doesn't have any internal records that mark right and wrong, it's actually a statistical model. ChatGPT's answer is essentially a probabilistic result collected from the corpus of Internet data that makes up GPT-3.
At that time, OpenAI also said that the most suitable positioning for ChatGPT should be a coding aid.
But imagine if OpenAI really taught ChatGPT to "think step by step like a human", it could completely replace some of the code writing work that requires rote memorization, and the consequence would be that some "junior" coders were completely eliminated.
Now, Silicon Valley executives are envisioning products where people with little programming experience describe their ideas and visions to AI, and then build anything they want, whether it's a website or a game.
A few days ago, Andrej Karpathy, Tesla's former head of artificial intelligence, just tweeted that "the hottest new programming language is English."
Use ChatGPT to debug and the effect is outstanding
This may not be a joke, such as the popular fried chicken ChatGPT, which has great potential.
A recent study from the University of Mainz and University College London found that ChatGPT not only does a great job of fixing bugs, but developers can also significantly improve their success rates through conversation.
According to the researchers, ChatGPT's debug performance is similar to that of common deep learning methods, CoCoNut and Codex, and significantly better than standard automated program repair methods (APRs).
Address: https://arxiv.org/abs/2301.08653
Solving code problems with ChatGPT is not new, but the unique ability to talk to humans gives it an edge over other methods and models.
To assess ChatGPT's debug performance, the researchers tested it using 40 pure Python problems from the QuixBugs benchmark and then manually checked that the suggested solution was correct.
Because of the randomness of the answers given by ChatGPT, the researchers tested each question 4 times individually.
Unlike other benchmarks for bot fixes, QuixBugs contains relatively small issues (low number of lines of code), which is ideal for use in conversational systems.
During testing, the researchers removed all comments and asked ChatGPT if the code was buggy and how to fix it.
For example, Figure 1 is an example of the BITCOUNT problem. Lines 1-2 are requirements for ChatGPT; Starting from line 4 is the wrong snippet.
For this example, we hope that the ChatGPT answer will solve the error in line 7 that nˆ = n - 1 should be replaced with n &= n - 1. In response, ChatGPT either gives a fixed piece of code or a description of how it should be changed.
The results showed that ChatGPT solved 19 of the 40 bugs, comparable to CoCoNut (19) and Codex (21), but the standard APR approach solved only 7 of them.
Of course, since ChatGPT and Codex both come from the same family of language models, it's not surprising that the number of problems solved is similar.
In addition, if we look closely at the results, we can also see that ChatGPT does not solve bugs in benchmarks every time. BUCKETSORT AND FLATTEN ALONE FOUND BUGS FOUR TIMES, WHILE OTHERS USUALLY ONLY SUCCEEDED 1-2 TIMES.
That said, users may need to try several times to get the right result when actually using it.
However, ChatGPT has a powerful advantage: we can interact with the system in the conversation, explain the problem in more detail, and thus get the correct answer.
The actual test results are also true.
After further dialogue with the model, the researchers managed to refresh the accuracy rate of ChatGPT to 77.5%, which is 31 out of 40 bug fixes, far surpassing SOTA.
At least, for now, it's entirely possible: developers won't have to write boilerplate code.
Instead, they can focus on areas such as complex application architecture or cybersecurity.
That said, while ChatGPT may do some programming work, such as writing generic functions or boilerplate code, it won't completely replace programmers. Because a programmer's job requires more than just writing code.
Being a programmer requires skill—being able to build programs, follow logic, and produce something bigger than the sum of its parts.
Coder: I "kill" myself
Obviously, ChatGPT is not the first "self-iterating" product made by coders.
Let's go to a row, those AIs who can write code.
Google's Pitchfork
Last November, rumors spread that Google was hatching a secret project that would train code through machine learning, compile its own code, fix bugs itself, and update itself.
According to people familiar with the matter, the project was originally developed by Alphabet's moonshot division, Division X, codenamed Pitchfork, and was transferred to Google Labs last summer.
According to internal sources, Pitchfork's role is to "teach code to write and rewrite itself."
It is able to learn different programming styles and write code according to those styles.
A Google employee said that Pitchfork was originally developed with the intention of building a tool to update Google's Python codebase to a new version.
AlphaCode: Beat 72% of programmers
In February 2022, DeepMind launched its "AlphaCode" system, which can generate code using artificial intelligence.
According to DeepMind, AlphaCode can rival humans.
DeepMind tested AlphaCode using 10 existing competitions hosted on the programming competition platform Codeforces, and it ranked in the top 54.3% overall, that is, it beat 46% of its entrants.
DeepMind claims that when tested using Codeforces, a programming competition platform, AlphaCode solved 34.2 percent of the problems in 1 million samples.
In addition, among the users who have participated in the competition in the past 6 months, AlphaCode ranks in the top 28%, which can be said to "beat 72% of human programmers"!
At the time, DeepMind pointed out that while AlphaCode is currently only available in competitive programming, it is clear that its future capabilities will not stop there.
It opens the door to the creation of tools that will make programming more accessible and one day fully automated.
Copilot: Code completion artifact
Further on, in 2021, GitHub and OpenAI jointly launched an AI programming artifact - GitHub Copilot.
As you enter code, Copilot automatically prompts for possible next snippets in the program, like an autocomplete robot trained to speak in Python or JavaScript.
Copilot can fill in the necessary blocks of code, as long as they are not particularly complex or creative, which is too useful for manual programming.
On June 22, 2022, Copilot was officially launched for the C-end with a price of $10/month or $100/year, and was provided free to student users and maintainers of popular open source projects.
Thousands of developers are now using Copilot. Code is written in more than a dozen of the most popular languages — up to 40 percent rely on it to build it.
GitHub predicts that developers will write up to 80% of their code using Copilot within five years.
Kevin Scott, Microsoft's chief technology officer, added: "We are sure that GitHub Copilot can be applied to thousands of different types of work."
However, less than 5 months after its release, Copilot has been taken to court by angry programmers for alleged infringement, claiming $9 billion.
And ChatGPT, which has learned "software engineering thinking", can beat them? At the speed of OpenAI, I'm afraid we won't have to wait long.
Editor: Aeneas is sleepy
Resources:
https://www.semafor.com/article/01/27/2023/openai-has-hired-an-army-of-contractors-to-make-basic-coding-obsolete
https://www.zdnet.com/article/chatgpt-can-write-code-now-researchers-say-its-good-at-fixing-bugs-too/