A recent study by Uplevel shows that no productivity increase is noticeable among developers when they use AI coding assistants.
Over three months, the output of 800 developers was compared with and without the use of Github’s Copilot coding assistant. It was expected that the team would be able to write more code and have to review and improve less. Surprisingly, Uplevel found no significant improvements, Techspot knows. In fact, thanks to Copilot, there were 41 percent more bugs in the code.
Coding and AI not a perfect match after all
AI coding assistants in general have yet to see much success, with workloads not dropping as hoped. The assistants often deliver code that is insecure or difficult to understand, causing developers to waste more time debugging than if they wrote the code themselves. Yet AI is often used to check or improve code. This is likely due to the rapidly growing supply of open source coding assistants and algorithms that are constantly learning.
read also
No increased productivity from AI assistants in coding
Companies like Innovative Solutions are pleased with the results, though, because there productivity is tripling thanks to tools like Claude Dev and Copilot. Clearly, coding with AI is still in its infancy and the success rate is uneven, but with the current rate of development, that could change very quickly.