Get a Good Return on Your AI Investments – O’Reilly

Final week, we had our first Infrastructure & Ops superstream of 2026, Platform Engineering in the Age of AI. Our audio system explored a spread of matters centered on supporting new AI workloads, every with distinctive infrastructure wants, unpredictable prices, and novel safety issues. Google Cloud’s Abdel Sghiouar took the viewers by way of what a great platform for AI appears to be like like, Cockroach Labs’ Jordan Lewis shared classes discovered rolling out a company AI platform, Syntasso’s Daniel Bryant outlined a three-layer mannequin for constructing a great platform, expertise chief Sarah Wells mentioned the significance of governance and find out how to make it extra manageable, and Thoughtworks’ Ben O’Mahony defined why evals ought to be a part of your observability story. You’ll be able to watch the highlights here.

The occasion concluded with a fireplace chat between Sam and Nathen Harvey, who leads the DORA crew at Google Cloud. DORA has been monitoring software program supply efficiency for over a decade, which implies they’ve watched plenty of expertise tendencies come by way of. Their middle of gravity has all the time been the identical query: How shortly and safely can a crew transfer change right into a operating manufacturing utility?

AI hasn’t modified that query, though it has made answering it a bit more durable. DORA lately launched its ROI of AI-Assisted Software Development report to indicate how AI is working for groups proper now, and the way that will or is probably not contributing to organizations’ backside strains. Nathen used the findings as a jumping-off level to dig into how AI is altering platform engineering and software program improvement as an entire.

The productiveness hole

Sam began by mentioning one of many greatest headline findings from DORA’S 2025 knowledge: Organizations noticed about 10% enchancment when it comes to precise code shipped to manufacturing programs. Though builders probably felt that they had been extra productive, that doesn’t routinely carry by way of to manufacturing. DORA’s knowledge reveals increased throughput alongside increased instability. In different phrases, groups are delivery extra however they’re additionally extra steadily rolling again modifications or implementing fixes. The beneficial properties on the particular person stage are actual (and 10% is a fairly good quantity), however these beneficial properties aren’t “the dramatic enhancements that you simply discover within the headlines.”

AI amplifies good processes (and unhealthy ones)

Nathen defined that AI is an amplifier and mirror that equally displays the great and unhealthy. On groups the place delivery change is already simple, AI tends to maintain issues operating properly. On groups the place getting grow to be manufacturing is painful, AI generates extra change and makes the prevailing friction extra acute. That mentioned, his learn on this end result is cautiously optimistic: “If the ache is extra acute, we perhaps will put money into addressing that ache.”

The rub is that the funding has to truly occur. Nathen famous that in lower-performing organizations, AI instruments typically arrive with a reset of expectations quite than an invite to repair the method: Right here’s your new device. Now we count on extra from you. Addressing this drawback means reframing the query “Does AI make individuals extra productive?” What we actually ought to be asking is “Underneath what situations will AI increase productiveness, and who’s chargeable for creating them?” And that falls on the group, not the expertise.

Verification isn’t a checkbox

Belief is a giant problem with generative AI. About 30% of DORA survey respondents belief AI output little or by no means. Round 46% belief it “considerably” (and Nathen is one among them). Regardless of all of the advances in generative AI, these instruments nonetheless make errors, and in case you’ve multiplied your capability to generate code with out doing something to scale your capability to confirm it, you’ve made your scenario worse, not higher.

Nathen referred to as this the verification tax, and it belongs in any sincere accounting of AI’s productiveness influence. Pipeline adaptation belongs there too: Is your supply pipeline match for function given the amount of change you’re now attempting to push by way of? These prices don’t present up within the headlines about 10x developer productiveness. They present up in your incident stories three months later.

DORA lately printed an ROI framework and calculator for AI-assisted software program improvement. Nathen was clear that there’s no common quantity to supply, and the calculator doesn’t faux in any other case. What it does is give groups a option to mannequin the true prices, together with the educational funding, the verification overhead, and the pipeline modifications required.

Context switching and burnout

With productiveness on the upswing, AI-induced burnout is changing into a severe concern. (Steve Yegge calls this the “AI vampire.”) DORA’s knowledge for 2025 confirmed that AI adoption wasn’t strongly linked with burnout, with the caveat that about 64% of DORA survey respondents mentioned they’d by no means labored in an agentic workflow. Each of these findings are more likely to change considerably in 2026.

Nathen highlighted one supply of burnout he expects to escalate as brokers grow to be the norm: context switching. As he identified, software program builders spent years arguing for protected focus time to do the deep work that requires them to keep up move. Agentic workflows at the moment are incentivizing those self same builders to voluntarily run a dozen or extra brokers without delay, forcing them to context-switch a number of occasions each hour. As he joked, “There’s loads of analysis that helps the concept that all of us really feel like we’re fairly good multitaskers and none of us are.” The results are coming, and we’re doing it to ourselves.

The cognitive debt query

Sam Newman introduced up the associated notion of “cognitive debt,” and specifically, Margaret-Anne Storey’s dialogue of it. (See “How Generative and Agentic AI Shift Concern from Technical Debt to Cognitive Debt” and “From Technical Debt to Cognitive and Intent Debt: Rethinking Software Health in the Age of AI.”) Right here’s how Storey explains the issue in her weblog publish:

Debt compounded from going quick lives within the brains of the builders and impacts their lived experiences and talents to “go quick” or to make modifications. Even when AI brokers produce code that could possibly be simple to know, the people concerned might have merely misplaced the plot and should not perceive what this system is meant to do, how their intentions had been applied, or find out how to presumably change it.

And as Sam famous, this compounds throughout groups and organizations. As builders more and more work in parallel with AI quite than with one another, they lose the shared understanding that comes from individuals constructing software program collectively. Kent Beck as soon as mentioned that “software design is an exercise in human relationships.” Agentic workflows are placing strain on that in methods we’re solely starting to see.

Nathen agreed cognitive debt is the place he’s most involved, and each your employees and your structure will endure for it. Understanding the ramifications of an architectural resolution you made eight months in the past takes years of operation to floor, and AI doesn’t assist with that in any respect.

Spend money on your platform now

Contemplating what makes some AI-assisted groups excessive performers, Nathen defined, “It’s not that you’re utilizing AI however how you’re utilizing AI.” This commentary led DORA to develop seven capabilities that, when mixed with AI adoption, result in higher outcomes. Nathen briefly ran by way of the record, ending on high quality inner platforms. And right here he made a declare about software program engineering funding that was, in his phrases, “a bit of bit wild”:

Each product engineer that you’ve in your group, each engineer that’s centered on constructing options proper now, ought to in all probability cease constructing options and give attention to the platform.

His argument is that platforms matter extra, not much less, in an setting the place AI makes it attainable for nearly anybody in a company to construct one thing. The individuals closest to prospects and enterprise issues can now generate working software program. What they’ll’t do is make sure that software program is sturdy, safe, and production-ready.

Nathen steered that the most effective leverage for software program engineering funding at this time could be constructing platforms that present these guardrails, that shift the complexity of production-readiness down into the infrastructure in order that anybody constructing on high of it will get the security internet without cost. He acknowledged that transferring each product engineer to platform work could be overkill. However the route of journey is actual. The platform can also be, as Newman identified, the place you convey determinism again right into a course of that AI has made extra nondeterministic.

That’s one thing we’ve been listening to lots right here at O’Reilly. The enlargement of who can construct doesn’t scale back the necessity for deep engineering experience. It modifications the place that experience is most precious, and platforms are a great reply to the place.

What DORA’s analysis tells us

The groups which might be doing properly are operating experiments, studying from them, and spreading these classes. The measure Nathen steered will not be what number of tokens you’ve consumed however what number of experiments you’ve run and the way properly you’re distributing what you’ve discovered.

The instruments are transferring quick sufficient that any group locking in a set coverage round particular instruments will discover itself caught. What you need is the capability to continue learning, which implies constructing the tradition and the processes that make studying seen and transferable.

All of DORA’s analysis is freely obtainable at dora.dev, together with the 2025 annual report and the ROI framework. The DORA Community offers an area for practitioners to work by way of these questions collectively. When you’re attempting to navigate any of this together with your crew, you could wish to spend a while there.

And if you wish to dive deeper into Nathen and Sam’s chat or discover the opposite classes, you’ll be able to watch the entire Infrastructure & Ops Superstream on the O’Reilly studying platform. Our subsequent occasion, on September 9, will cowl agentic observability. Register for free here, and take a look at all the opposite free live events on O’Reilly.