The enterprise AI challenge nobody solves with code generation alone

Presented by SAP

Generating code with AI is fast, but getting that code to run reliably inside a large enterprise, integrated with live systems, governed for compliance, and maintainable over years requires foundational work that most organizations underestimate.

While 81% of all organizations have a detailed strategy, only 12–16% reach AI‑driven execution, says SAP's Michael Ameling, CPO of SAP Business Technology Platform, and the reasons rarely come down to the quality of the generated code.

"Across industries, enterprises that have invested heavily in AI tooling are hitting a wall when generated code meets the reality of their existing environments, because generating code and operationalizing it are not the same problem," Ameling says.

There are specific requirements for deploying AI-generated logic at enterprise scale: what data and integration readiness actually look like, how governance works when AI agents move from producing recommendations to executing workflows, and how development teams are changing their role as AI takes over more of the coding work.

Why AI code generation fails in enterprise production environments

The productivity gains from AI code generation are real and well-documented, but the ease of prototyping has given many organizations a misleading sense of how far along they actually are.

"Generating code is one thing," Ameling says. "Enterprise customers, including multinationals and large organizations, need to ensure there are no compromises in compliance or security. Code that runs reliably for ten or twenty years, as it does at many of SAP's largest customers, also has to be maintained, patched, and understood by whoever inherits it. Life cycle management, in other words, does not generate itself."

The issue is rarely the generation quality. Teams build something compelling, then discover they lack access to the data it depends on, or the integrations it assumes, or the permissions required to run it in a real environment. The problem is essentially that AI amplifies an organization's existing data and process maturity, but it can't substitute for it.

This dynamic intensifies as AI moves from producing code to executing actions. Latency, cost, and system load all increase when logic runs continuously against live data rather than rendering a one-time output. The performance requirements of an autonomous agent operating across a multinational's transaction systems are categorically different from those of a developer copilot.

How to connect AI-generated logic to fragmented enterprise systems

The architecture challenge that most enterprise AI projects underestimate is integration. Real enterprise environments are not clean slates: they combine cloud systems, legacy on-premise infrastructure, fragmented data stores, and dozens of business applications that were never designed to talk to each other. Getting AI-generated logic to operate reliably across all of them requires a layer that unifies data access, process context, and governance, and it has to be in place before any agent starts executing. And organizations that see AI as a reason to defer infrastructure modernization are making a mistake.

"The question is not whether to modernize or not. Of course you need to modernize," Ameling says. "But the value you get on top of this is much higher with AI. Federated data access and harmonized process layers are not alternatives to upgrading a fragmented landscape, they're what make the upgrade worthwhile."

At the platform level, this translates into a set of practical requirements: structured data integration, end-to-end process visibility, and the ability to discover and connect to APIs across both modern and legacy systems. SAP's approach with the Business AI Platform draws on tools including its Joule Studio, Integration Suite, Business Data Cloud, and SAP AI Agent Hub enterprise architecture layer to provide that context. The goal is to give AI-generated logic accurate, current knowledge of what a business is doing and how, rather than just access to raw data.

AI agents handle large challenges by dividing them into smaller, autonomous tasks, with each agent responsible for a specific domain, and all coordinated toward a shared outcome. A financial close, for example, involves dozens of discrete sub-processes. Agents handling each task in parallel, within defined constraints, can compress cycle times dramatically, but only if the underlying systems they interact with are coherent and accessible.

The governance and oversight that AI agents require in production

When AI moves from assistant to operational actor, the governance questions loom large, because agents that trigger workflows, update records, and interact with live business systems need the same accountability framework that applies to human employees, i.e., identities, defined privileges, and auditable behavior.

There are two distinct models:

Principal propagation, where an agent acts on a user’s behalf, inheriting that user’s permissions and scope.

System-triggered agents, where the agent operates under its own identity and role-defined privileges, functioning more like an automated HR role than a personal assistant.

Both models require the same underlying infrastructure: an agent hub where operators can see which agents exist, what APIs they can access, and what they are authorized to do. Observability also needs to be operationalized correctly for AI, combined with both technical and business evals.

"In production, openness is very important," Ameling says. "We use OpenTelemetry as a framework, so we can integrate with other solutions, for end-to-end observability of the tool, third-party agents and the like."

On top of that, standard technical evals, which test whether an agent produces consistent outputs, are necessary but not enough. Business evals assess whether an agent is actually moving the performance indicators it was deployed to improve, but it has to work end-to-end.

Where the testing happens is equally important. The traditional software development cycle across dev, test, and production environments breaks down when a model produces different outputs depending on whether it is running against test data or live data. Getting to trustworthy AI in production means accepting that validation looks fundamentally different from what engineering teams have practiced for decades, with live environment testing, even A/B/C testing to ensure outcomes are reliable.

How AI-driven code generation is changing software engineering roles

The role of the developer is not disappearing in this environment, but its center of gravity is shifting. The productivity multiplier is significant when developers can run multiple coding agents in parallel across open terminals, each working on a separate problem and each taking several minutes to complete. But it introduces a new kind of cognitive demand, because humans have to stay in the loop. That means tracking context across concurrent workstreams, evaluating outputs that range across large codebases, and making architectural judgments that no agent can be trusted to make alone.

"The more specific and complete the prompt, the less intervention is required, and developers are learning that bringing more context upfront pays dividends in reduced back-and-forth," Ameling says. "But the output still needs to be understood, not just accepted."

The competitive edge will remain intellectual property, not tooling. The companies that pull ahead will be those that most effectively encode their domain knowledge into the systems they build.

"A manufacturer's process expertise, a financial institution's risk logic, a logistics firm's routing intelligence, these are the assets that AI can accelerate, but only if the organizations that hold them do the work to make them accessible and usable," Ameling says. "Protect that, and apply AI to accelerate your differentiation."

Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. For more information, contact [email protected].