GitHub, a platform that provides code storage and collaboration services, recently launched Copilot, an AI programming assistant threatened by lawsuits, that has already attracted many paying users. To find out if developers are already using AI programming tools in their work, GitHub commissioned Wakefield Research to conduct a survey of 500 developers at large U.S. companies.
The survey results surprisingly show that developers are already using AI programming tools widely and believe that these tools can bring significant benefits. Inbal Shani, GitHub's chief product officer, said the purpose of the survey was to learn more about how developers are doing in terms of productivity, collaboration, artificial intelligence and enterprise environments.
Usage and benefits of AI programming tools
According to the survey, 92 percent of developers at large U.S. companies say they use AI programming tools in their work or personal time, and 70 percent of them say they see significant benefits from using them.
Interestingly, the survey also revealed that developers believe that AI programming tools can improve the quality of the code they create.
"Developers say AI programming tools can help them meet existing performance standards, improve code quality, speed up output, and reduce production-level incidents." The survey report.
However, survey respondents may not be aware of some academic research to the contrary. For example, one study found that code generated by AI assistants like ChatGPT was "well below the minimum security standards that apply in most cases." Another study found that code generated by Copilot was more prone to security vulnerabilities than code without the help of artificial intelligence, while developers mistakenly believed that machine learning tools would produce fewer errors. A third study found that code generated by Copilot had a 40% chance of having security flaws.
The impact of AI programming tools on developer evaluation criteria
In the case of using AI programming tools, developers seem to want to reduce the importance of code quality as an evaluation metric. When asked how they are currently being evaluated, the programmers replied: code quality (40%); Time required to complete tasks (34%); Number of production accidents (34%); Number of lines of code written (33%); Number of bugs or issues resolved or fixed (33%).
Assuming AI programming tools, these developers prefer to be evaluated by the following criteria: code quality (36%); Time required to complete tasks (36%); Number of production accidents (33%); Number of lines of code written (33%); Number of pull requests (32%).
This seems to mean that developers are more inclined to pursue speed than quality. And, on the self-interested side, an equal percentage of developers say they currently evaluate themselves by the number of lines of code they write, and they believe that this metric should continue to be used even with AI assistants.
GitHub's survey suggests that this may need to be reconsidered: "As the use of AI tools in software development increases—which tends to increase the amount of code—engineering leaders will need to ask themselves whether measuring productivity and output in terms of code volume is still the best approach." ”
The impact of AI programming tools on developer collaboration and satisfaction
Many surveyed developers (81%) also see AI as a way to improve collaboration within teams and organizations. When asked what aspects teams using AI tools should focus on, they replied: code reviews (45%); Security review (45%); plans (43%); pair programming (43%); Solution design (42%).
When asked how AI tools help them in their day-to-day work, developers replied: improve your programming language skills (57%); Increase your productivity (53%); Focus on building or creating rather than repetitive tasks (51%); Prevent burnout (41%).
The survey concludes: "Developer satisfaction, productivity, and organizational impact are all expected to benefit from AI programming tools – which will have a significant impact on the overall developer experience." ”
It's like you hear how much writers love automatic error correction and spell checking.