How much more productive are developers using AI coding tools? Recently, there has been a lot of speculation that AI makes developers 2x, 3x, or even 5x more productive. One report predicts a tenfold increase in developer productivity by 2030.
The irony, however, is that the engineering community has, for the most part, not been able to agree upon a universal way to measure engineering productivity. Some have even rejected the idea altogether, arguing that most metrics are flawed or imperfect. Most of the claims around AI improving productivity today are qualitative — based on surveys and anecdotes, and not on quantitative data.
How can we make judgments about AI without first agreeing on how to measure productivity? If we learned anything from the remote work experiment, it’s that we floundered without data to inform our decisions — shifting back and forth between office, remote, and hybrid strategies based on dogma and ideology instead of data and measurement.
We’re on a path to repeat ourselves with AI. To move forward, we must first understand and quantify its impact.
The risk of falling behind
The current hype around AI may give some of us reason to pause — due to the unknown impact to quality, the potential risk of plagiarism and other factors. The most cautious companies have entered a holding pattern, waiting to see how it all plays out.
For tech-enabled businesses, however, the risk of falling behind is existential. AI is a double accelerant, impacting both what and how companies build. Companies that invest in AI today have the potential to double dip by bringing to market not only new AI-powered products, but also products to market faster and more cheaply.
Most companies have been focused on the what, but AI could be the driver for the how, creating the 10x or even 100x engineering team. Companies that figure out how to quickly cross the chasm — by optimizing AI tools in the most efficient and impactful way — and reach the plateau of productivity faster will benefit from a head start for years to come. The risk of doing nothing is too high.
Understanding the trade-offs
To someone with a hammer, everything looks like a nail. So, too, with AI.
According to a recent GitHub report, the top benefit of AI coding tools cited by developers was improving their coding language skills. Another key benefit is automating repetitive tasks, like writing boilerplate code. A recent experiment by Codecov showed that ChatGPT performs well at writing simple tests for trivial functions and relatively straightforward code paths.
