AI code accelerates production failures and spending, study finds

AI + ML

CloudBees survey exposes verification gap

The rapid adoption of AI-generated code is driving production failures and higher costs for enterprise customers.

Eighty-one percent of enterprise technology leaders among more than 200 surveyed reported an increase in production issues linked to AI-generated code, according to a study published by enterprise software delivery biz CloudBees.

Sunil Gottumukkala, CEO of Averlon, an agentic vulnerability remediation biz, told The Register in an email that these issues tend to refer to functionality bugs, performance issues, availability problems, and security vulnerabilities rather than CI/CD failures. 

“These are issues that surface after code has already been deployed to production, which means the code passed every review and deployment gate and still broke things,” said Gottumukkala. “When failures happen post-deployment, it signals that the validation process itself isn’t keeping pace with what AI is producing.”

Yet 92 percent of respondents expressed confidence that their code was production-ready before it shipped.

Jacob Krell, senior director of secure AI solutions and cybersecurity at Suzu Labs, told The Register in an email that the report does not isolate what specifically failed at these organizations.

“It spans functional defects, security vulnerabilities, and compliance violations that reach production because governance and validation have not scaled with output,” he said. “The same study found 69 percent citing security vulnerabilities and 63 percent citing compliance issues introduced by AI generated code specifically.”

Krell said what ties them together is the verification gap.

“AI generates code faster than teams can validate it,” he said. “Seventy percent of respondents now say test suite maintenance is a larger burden than writing code itself. These are not system crashes in the traditional sense. They are the full spectrum of what reaches production when volume outpaces the capacity to verify quality, security, and compliance before deployment.”

Respondents said 61 percent of their organizations’ code has been generated by AI or has come into being with AI assistance. And 64 percent of the engineering organizations involved say AI is widely or fully integrated into their workflows.

The result is that more than half (52 percent) of those surveyed report an uptick in software development output. And while 68 percent of organizations appear to be convinced AI is delivering business value, only 31 percent of AI-related spending can be linked to specific business results. In 36 percent of organizations, AI spending is tracked without measuring the return on investment or isn’t tracked at all.

With more code comes more cost from infrastructure spending, in the form of increased CI/CD, testing, and security scanning. Some 54 percent of respondents said CI/CD infrastructure spending has risen significantly in the past 12 months, and 53 percent flagged rising testing, security, and deployment costs. 

Only 45 percent of respondents say these costs are predictable quarter to quarter. Yet relatively few organizations have taken steps to control AI spending: 27 percent report quotas or limits on token usage, while just 18 percent have automated spending controls.

And this is a problem without ownership. Just 12 percent of organizations have dedicated AI governance. For 46 percent, the buck stops with the CTO or VP of engineering when there’s a production failure. For 32 percent, blame falls on the engineering lead or team associated with the tool that messed up the code. For 7 percent, the developer who shipped the pull request takes the heat.

It may be tempting to take comfort in the fact that 93 percent of respondents say their organization has a formal process for reviewing and releasing AI-generated code. But keep in mind that only 56 percent of survey takers say those processes are always enforced. ®