Optimizing application architectures for AI: From monoliths to intelligent agents - Part 1

When we started designing our Red Hat Summit 2025 session with Mark Cheshire and Bernard Tison, Optimize your architecture for the next wave of AI-infused applications, we faced two realizations.

One was that in order to talk about what type of architectures are required to develop and scale AI applications, including agents and agentic AI, we needed to provide some context about the evolution of architectures. This was important because we believe that there is no need to throw everything out and start afresh for AI apps. Rather, our premise was that you will succeed by thinking about this as an evolution of your existing application investments.

The other realization was that given how fast things are changing and evolving in AI, we would have to talk about the most advanced publicly available AI technologies and approaches to AI application development and integration. Hello, Llama stack and MCP.

We solved this by realizing that enterprise software architecture has always evolved in response to pressure. First it was scale. Then maintainability. Then integration. And now, it’s artificial intelligence. And we thought Darwin and his “theory of evolution” would be a good guide.

This article is a look at where we’ve been (and where things are headed) as generative AI (gen AI) and large language models (LLMs) reshape the design of enterprise applications. From monoliths to microservices to multi-agent systems, the way we build is changing again. Moreover the rate of change is faster in this cycle than many that have preceded. It is not an option to wait until the new paradigm has stabilized—we need to embrace the fact that change will continue. This means that the need for clear protocols, open interoperability, and platform flexibility remain constant.

It is all going very fast, and technology transitions can be hard

Gen AI has experienced the fastest adoption rate in technology history. From the moment ChatGPT was made public in November 2022, it took just 5 days to get to a million users and just a year to get to 100 million users. Its current pervasiveness, the desire of users to incorporate it into their daily jobs, and the potential enterprises see in AI changing everything from selling to manufacturing to providing services is putting a lot of pressure on IT teams. 

However, there is a significant gap between expectations and reality.

Boston Consulting Group did a survey with almost 2,000 business leaders this year where about 75% of them said AI is one of their top 3 priorities. But even so, only 25% of these companies were seeing real value from AI today. 

Also, according to McKinsey, more than 90% of companies are increasing their spending on AI, but only 1% believe they have reached a "strong" or "mature" level in their AI efforts. This tells us that while interest is high, most companies are still early in their AI journey.

All of this shows that while AI is a priority, success is not guaranteed. We must be prepared to face these challenges and guide companies through this time of change. It is imperative that investments move beyond treating AI projects as siloed experiments in incubation centers, to add value within the enterprise application fabric.

And it all starts with how AI is incorporated into the way business works, which is of course based on the apps they use to run their business, where their authentic competitive advantage lies.

From monoliths to intelligent agents

In the early days, enterprise applications were built as monoliths – all the logic lived in one place. The user interface, the business logic, even the database layer was all tightly coupled, often written in a single language, and deployed as one unit. This made development straightforward, but scaling, updating, or even understanding the application became a challenge as complexity grew.

It worked for small systems, but it was hard to scale or update.

Later, to improve scalability and maintainability, we moved to layered (or N-tier) architectures. In this model, the presentation layer was separated from the logic layer and different types of databases were introduced to support specialized needs. This change gave teams more flexibility in development and deployment.

As the demand for even greater scalability increased, especially with growing numbers of users and the needs to scale up applications in cloud environments, we transitioned to microservices. These are small, independent services that handle specific tasks. Microservices can be developed, deployed, and scaled separately, allowing organizations to respond more quickly to change and serve users more efficiently. This usually required application modernization and adoption of new cloud native frameworks like Quarkus.

But now, another shift is underway as AI enters the architecture. Gen AI, especially in the form of LLMs like GPT, Claude or Gemini, is changing how we think about software components. We're not just wiring up services anymore. We’re deploying agents.

These intelligent agents are more than just another microservice. They're runtime components with advanced capabilities. They can interpret natural language, make decisions, interact with APIs and tools, and generate responses or actions based on context. 

Under the hood, an agent typically consists of two parts:

• A software application that handles the agent's lifecycle and tool integrations
• An LLM or small language model (SLM) that powers the agent’s reasoning and language capabilities. These SLM gen AI models can be safely trained on your own data using tools like RHEL AI and InstructLab.

This pairing introduces a new design pattern: one where application workflows aren't hard-coded but inferred dynamically by agents based on real-time context. But it also introduces new requirements, especially around integration.

But I really like Java! Do I need to use Python?

You don't need to change your current technology setup to start working with agents. There is no need to switch to Python just because many open source AI tools are built with it. If your team is already comfortable using Java or another advanced programming language, you can keep using it. Most modern language models provide REST APIs, so your Java-based microservices can connect to them just like Python applications do. The most important thing is to have clear communication rules and a reliable way to manage the system.

Also, using agents does not mean you have to stop using microservices. In fact, microservices are still a strong choice for building systems, and they work well with agents. You can think of each agent as a small service that works on its own in a specific area. Agents can run in separate containers, grow with demand, and be managed by themselves—just like microservices. They can also work together, where one agent activates or helps other agents to complete more complex tasks.

This setup helps companies make changes step by step. You do not need to rebuild everything to start using agents. You can simply add them to your current system. The agent layer becomes a smarter way to handle tasks and interactions, giving your system more flexibility and intelligence without starting over.

A new layer of integration: Model Context Protocol

As agents multiply across an enterprise, the need for coordination grows. REST and GraphQL work well for traditional APIs, but they don’t solve for dynamic tool use, context passing, or cross-agent orchestration.

That’s where Model Context Protocol (MCP) comes in. Introduced in November 2024 by Anthropic and rapidly adopted by communities (including Red Hat where we are integrating Llama stack with MCP, which will provide users with standardized APIs for building and deploying AI applications and agents), MCP is an emerging standard that acts as a coordination layer between agents and tools.

MCP allows agents to:

• Discover which tools are available in their environment
• Select the appropriate tool for a given task
• Generate input parameters based on context
• Delegate execution to a system that understands how to use that tool

And best of all, it does this without developers having to write glue code for every possible integration.

In our session at Red Hat Summit we illustrated this with a few examples. One of these involved a user asking an agent to “show customer order history.” The LLM doesn’t need direct database access. Instead, it uses MCP to identify a registered tool (e.g., getOrderHistory), construct the request, and delegate the task to the application. The result is returned to the agent, which can generate a natural-language response or trigger the next step in a workflow.

Intelligent applications, powered by agents

The result of combining microservices, agents and MCP is a new kind of architecture: one where logic is no longer rigidly defined up front but composed dynamically at runtime based on context.

This flexibility means applications can scale not just in terms of user volume but in terms of complexity, adaptability and personalization. Agents can tailor workflows to specific users or tasks. And because MCP is model-agnostic, this approach works with any programming language or AI model. Many teams are building agents in Java (with frameworks like Quarkus), and connecting to LLMs via APIs or MCP endpoints.

What comes next

We’re in a transitional phase. The previous generation of architecture isn’t going away—it’s expanding. AI agents don’t replace microservices—they layer on top of them, offering new opportunities to simplify user experiences and automate decision-making.

If Darwin was right, adaptability wins. The question for enterprise teams is: Can your architecture evolve to meet what comes next?

Red Hat is investing heavily in open standards, interoperable tooling, and hybrid cloud platforms to help make that transition easier. Whether you're exploring agents with Llama Stack, inferencing Llama 4 with vLLM, deploying inference at scale with Red Hat AI Inference Server, or connecting applications using MCP, the goal is the same: Let developers focus on solving problems, not wiring components.

Stay tuned for the second blog of this series, where we will go through a practical enterprise example showcasing AI agents development and integration to support human decisions and speed up processes.

And, in the meantime, read about how to navigate the generative AI landscape, Red Hat’s approach to agentic AI and tooling up your LLM with Apache Camel on OpenShift.