Application Structure in the AI Era¶

AI does not invalidate established software-architecture principles. It increases the value of explicit module boundaries because model providers, retrieval systems, tool protocols and agent runtimes change faster than the business domain they support.

A useful default is:

Stable application/domain core
          ↑ ports
          │
Volatile AI/infrastructure adapters

The goal is not to hide AI. The goal is to prevent provider/framework details from becoming the architecture of the application.

Start with business capabilities, not AI technologies¶

A weak package structure often mirrors vendors:

openai/
qdrant/
mcp/
prompts/
agents/

This describes technologies, not the application.

A stronger top-level structure usually starts with business/use-case boundaries:

application/
├── support/
├── billing/
├── code_review/
├── knowledge/
└── shared_runtime/

Each module may internally use LLMs, retrieval or tools without exposing those implementation details to every other module.

For example:

support/
├── domain/
├── application/
├── agent/
└── infrastructure/

The support module owns the support use case. The fact that one application service uses an LLM does not mean the whole application becomes an agents/ package.

Modular monolith as a strong default¶

A modular monolith is one deployable application with strong internal module boundaries.

┌──────────────────────────────────────┐
│            one deployment            │
│                                      │
│  Support   Billing   Knowledge       │
│     │         │          │           │
│     └──── Agent Runtime ─┘           │
│                │                     │
│         Infrastructure Adapters      │
└──────────────────────────────────────┘

Advantages for many early and medium-size AI applications:

simple local development,
easy transactions where needed,
low operational overhead,
direct typed calls between modules,
easier refactoring while product boundaries are still evolving,
no distributed tracing/network retries for every internal interaction.

The key word is modular. A monolith with arbitrary imports and shared mutable state is not the goal.

When microservices are justified¶

Service extraction should answer a concrete problem.

Good reasons include:

a GPU-heavy worker needs independent scaling,
ingestion runs have a different availability profile,
a security-sensitive tool executor needs stronger isolation,
another team owns a bounded context independently,
a long-running agent runtime requires independent deployment,
data residency/tenant boundaries require physical separation,
workload characteristics differ enough to justify separate infrastructure.

Weak reason:

"It uses AI, therefore it should be a microservice."

Distributed systems add queues, retries, partial failure, tracing, consistency and deployment complexity. That cost should purchase something real.

Hexagonal Architecture / Ports and Adapters¶

Hexagonal Architecture is especially useful for AI because many dependencies are volatile.

The application defines ports representing what it needs:

class ReasoningPort(Protocol):
    def decide(self, request: DecisionRequest) -> Decision: ...

class RetrieverPort(Protocol):
    def search(self, query: RetrievalQuery) -> list[Evidence]: ...

class TicketPort(Protocol):
    def create(self, command: CreateTicket) -> TicketId: ...

Infrastructure provides adapters:

ReasoningPort
├── OpenAIAdapter
├── AnthropicAdapter
└── LocalModelAdapter

RetrieverPort
├── QdrantAdapter
└── PgVectorAdapter

TicketPort
├── JiraAdapter
└── ZendeskAdapter

The application depends on its own abstractions, not on vendor SDKs.

Application Core
      │
      ▼
     Port
      ▲
      │
Infrastructure Adapter

The dependency points inward.

Do not create meaningless ports¶

Dependency inversion does not mean wrapping every library automatically.

A useful port expresses an application capability, not a one-to-one rename of a vendor API.

Weak:

class OpenAIChatCompletionPort:
    def create_chat_completion(...)

This leaks the provider abstraction inward.

Better:

class ClassificationPort:
    def classify_ticket(...)

class AgentDecisionPort:
    def choose_next_action(...)

Sometimes a generic ModelGateway is appropriate at the runtime layer, but business modules should still consume domain/use-case-level capabilities where possible.

Clean Architecture¶

Clean Architecture reinforces the same direction:

Domain
  ↑
Application / Use Cases
  ↑
Agent Runtime / Interfaces
  ↑
Infrastructure / Providers

The inner layers should not know:

OpenAI request object shapes,
vector-store client types,
MCP transport details,
HTTP framework request objects,
queue vendor message classes.

Translate at boundaries.

LLM-facing DTO vs domain model¶

Do not expose domain entities directly as model schemas merely because structured output supports JSON schemas.

Prefer:

LLM Output DTO
      ↓ validate/map
Application Command / Value Object
      ↓
Domain

Example:

RefundDecisionDTO
  reason
  recommendation
  confidence
        ↓
RefundProposal
        ↓
RefundPolicy

The LLM can produce a proposal. The domain owns whether a refund is legal/allowed.

DDD and bounded contexts¶

DDD remains useful because AI often spans several business domains.

Imagine one agent can answer billing questions and technical support questions. That does not mean billing and support should share one AI domain model.

Billing Context
  invoices
  payment state
  refund rules

Support Context
  cases
  troubleshooting
  escalation rules

An agent runtime may orchestrate capabilities from both contexts through explicit application interfaces.

The runtime should not become a new "god bounded context" owning every domain concept.

Shared runtime vs business agent modules¶

A clean split can look like:

shared_runtime/
├── orchestration
├── model_gateway
├── context
├── policy
├── run_state
└── observability

support/
├── domain
├── application
├── skills
└── adapters

billing/
├── domain
├── application
├── skills
└── adapters

The shared runtime knows how to execute agentic work. Business modules know what their capabilities mean.

This reduces two opposite risks:

duplicating runtime mechanics in every module,
putting all business logic into one generic agent framework.

Where RAG belongs¶

RAG is normally a subsystem/capability, not the application architecture.

Use Case
   ↓
KnowledgePort
   ↓
Retrieval Service
   ├── query transform
   ├── retriever
   ├── reranker
   └── provenance
        ↓
Vector/Search adapters

The domain should not know vector distance metrics or chunk IDs unless they are genuinely part of the domain.

Where MCP belongs¶

MCP is an integration/protocol boundary.

Agent Runtime
    ↓ Tool/Resource Port
MCP Client Adapter
    ↓
MCP Server

Business logic should depend on a capability such as ReadRepository, not on the fact that the implementation happens to use MCP.

MCP may be extremely useful operationally, but it should not become the domain model.

Sync vs async boundaries¶

Not every AI call needs a queue.

Use synchronous execution when:

latency is acceptable,
the request is short-lived,
the caller expects an immediate result,
failure handling is simple.

Use asynchronous execution when:

runs are long,
work fans out,
retries may take minutes,
ingestion is heavy,
external dependencies are slow/unreliable,
pause/resume or human approval is required.

Example:

POST /analysis
    ↓
CreateRun
    ↓
Queue
    ↓
Agent Worker
    ↓
Checkpoint Store
    ↓
Result / Event

Queues are a runtime boundary, not an excuse to turn every module into a service.

Event-driven integration¶

Events are useful when one module should react without tight synchronous coupling.

DocumentIngested
      ↓
Indexing handler
      ↓
Embedding + Search index

Or:

AgentRunCompleted
      ↓
Audit / analytics / notification

Prefer domain/application events with stable semantics over infrastructure-shaped events such as OpenAIResponseReceived unless that event is genuinely operational.

Preventing AI concerns from leaking everywhere¶

A useful rule is:

The deeper a module is toward the domain core, the less it should know about models, prompts, tokens and providers.

Provider-specific concepts should usually stop at adapters/runtime boundaries.

For example, the domain should not contain:

model_name
prompt_template_id
temperature
vector_dimension
mcp_server_url

unless one of those is genuinely a business concept.

Example modular structure¶

src/
├── support/
│   ├── domain/
│   ├── application/
│   ├── skills/
│   └── infrastructure/
│
├── knowledge/
│   ├── application/
│   ├── retrieval/
│   └── infrastructure/
│
├── agent_runtime/
│   ├── orchestration/
│   ├── state/
│   ├── context/
│   ├── policy/
│   └── ports/
│
└── infrastructure/
    ├── llm/
    ├── persistence/
    ├── messaging/
    └── mcp/

This is only one possible structure. The key idea is dependency direction and ownership, not folder names.

Common anti-patterns¶

AI layer owns all business logic¶

Everything routes through one generic AgentService and domain modules become thin data access layers.

Vendor-driven architecture¶

The repository structure mirrors provider SDKs instead of business capabilities.

Premature microservices¶

Each skill/tool becomes a service before independent scaling/ownership/security requires it.

Shared prompt dumping ground¶

A global prompts/ directory becomes an invisible dependency graph used by unrelated modules.

Infrastructure types crossing boundaries¶

Vendor response objects and vector DB records travel directly into application/domain code.

Agent bypass path¶

Normal application use cases enforce rules, while an agent writes directly to the database or external API.

Decision checklist¶

For each AI subsystem ask:

Which business capability owns it?
Is it domain logic, application orchestration, runtime infrastructure or an adapter?
What stable port does the caller actually need?
What provider-specific details can remain outside the core?
Does it truly need a separate process/service?
If asynchronous, where are idempotency and state ownership handled?
Can the AI path bypass existing business rules?

Takeaways¶

AI does not replace software architecture; it makes boundaries more valuable.
Modular monolith first is often a strong default, not a universal law.
Use microservices for concrete scaling, ownership, isolation or lifecycle reasons.
Hexagonal/Clean Architecture can isolate volatile LLM/RAG/MCP/provider dependencies.
Organize around business capabilities and bounded contexts, not vendor names.
Keep LLM DTOs separate from domain models.
RAG, MCP and queues fit behind explicit application/runtime boundaries.
Never create an agent path that bypasses the domain/application rules used by the rest of the system.