Thoughts On Using AI To Inform Architecture Decisions

As software architects, we're living through an unprecedented shift in how engineering decisions are made. Large Language Models have become ubiquitous in our workflows—and for good reason. They're powerful tools for code assistance, idea generation, and exploring technical possibilities. But as with any powerful tool, knowing when and how to apply it matters enormously.

This post isn't about resisting AI or dismissing its value. It's about recognizing a fundamental mismatch: **LLMs optimize for plausibility, not operational reality.** And in architecture decisions, that distinction is everything.

The Plausibility Trap

Ask an LLM, "What's the best modern architecture for a Java REST API?" and you'll receive an answer that sounds sophisticated, current, and technically sound. The model will confidently recommend patterns, frameworks, and approaches that are indeed used successfully across the industry.

But here's what the model doesn't see:

• Your production reliability history

• Your operational complexity and deployment pipelines

• Your team's skill distribution and on-call burden

• Your existing technical debt and how systems interact under load

• Your customer requirements and what "uptime" actually means to your business

These invisible factors represent 90% of any real architectural decision. Without them, even the most technically correct answer is incomplete.

The Pattern We've Seen Before

This isn't unique to AI. It's a cycle that repeats with every new technology wave:

1. A powerful new tool appears

2. Engineers (understandably) get excited and overgeneralize its applicability

3. "Newer" becomes conflated with "better"

4. Existing systems get labeled "legacy" (as if that's inherently bad)

5. Stability and operational knowledge become undervalued

What makes AI different is that it can **validate these assumptions**. When you ask, "Is our architecture outdated?" it may tell you there are more modern patterns available. That feels like confirmation. It feels like evidence. But it's not the full picture.

Stability as a Feature, Not a Failure

Here's an uncomfortable truth that doesn't show up in AI training data: **A system that customers trust, that engineers understand deeply, that deploys predictably, and fails gracefully is far more valuable than a theoretically superior architecture.**

A production system that's been running for years is often quietly optimized in ways no model can comprehend—edge cases handled, failure modes understood, operational runbooks refined through real incidents. This institutional knowledge is invisible to AI but invaluable to your business.

Stability isn't stagnation. It's an achievement. And architectural decisions should reflect that.

The Evidence-First Framework

This doesn't mean we avoid change. It means we demand clarity before we commit. Here's the principle:

**Architectural change should only happen when the improvement is large enough to offset the operational and migration risk.**

Before any significant architectural shift, we need answers to three questions:

1. What specific problem are we solving?

• Not "this pattern is newer" or "AI recommended it"

• Real, measurable pain: latency issues, deployment failures, observability gaps, scaling bottlenecks, developer productivity problems

2. What measurable improvement do we expect?

• Quantify it. How much faster? How much more reliable? How much easier to maintain?

• If you can't measure it, you can't validate it worked

3. What is the migration cost and risk?

• Cost of rewriting services

• Time spent retraining engineers

• New operational runbooks and monitoring

• New failure modes to discover (the hard way)

• Impact on on-call support and incident response

Until you can answer all three, you don't have a decision—you have an experiment. And experiments are great, but they shouldn't be confused with validated architectural choices.

Operational Memory: The Missing Context

LLMs are pattern-matching machines trained on broad datasets. They almost never consider:

• Operational load and real-world failure modes

• Deployment complexity and rollback strategies

• Human factors and team capacity

• The cost of context-switching in production systems

AI has no operational memory. It doesn't remember the 2 AM incident where that "simple microservice split" cascaded into a regional outage. It doesn't recall the three months it took to retrain the team on the new observability stack. It has no concept of technical debt compounding over time.

This is why AI is excellent for exploring options and generating ideas—but architecture decisions must be validated against our operational reality.

The Experience Gap

There's a generational pattern worth acknowledging without judgment:

• Engineers with less tenure often see modernity = quality

• Engineers with more tenure often see predictability = quality

Both perspectives have value. Fresh eyes spot inefficiencies that long-tenured engineers have normalized. Experienced architects see risks that newer engineers haven't lived through yet.

The key is creating dialogue between these perspectives—not letting either one dominate unchecked.

The Right Questions to Ask

When faced with a proposal for significant architectural change (especially one validated by AI), shift the conversation from ideas to evidence:

For the proposer:

• "What production problem does this solve for us today?"

• "What would the migration involve, and how would it affect uptime or existing deployments?"

• "How does this affect our tracing and observability?"

• "What happens if we scale to double our current load with this new architecture?"

• "How do we roll back if this fails?"

These aren't blockers. They're guardrails. Good architecture decisions should survive scrutiny.

For the skeptic:

• "Let's make sure any changes we make improve things for both the team and our customers—can we quantify that improvement?"

• "I like that you're thinking about modern patterns. Let's make sure we're aligning them with our operational reality before making changes."

• "That sounds worth exploring—can we map out the migration steps, the risk/reward, and how the new solution interacts with our existing deployment, tracing, and logging?"

The tone matters. We're not shutting down innovation; we're protecting our team, our customers, and the stability they depend on.

Good Architecture Embraces Constraints

Architecture quality isn't measured by how modern it looks on a diagram. It's measured by:

• Uptime and reliability under real-world conditions

• Operational simplicity and mean time to recovery

• Developer velocity and ease of onboarding

• Failure recovery and graceful degradation

Good architecture focuses on constraints, not tools. It addresses actual pain, not bias introduced by an AI recommendation or industry hype cycle.

A Healthy Path Forward

Here's the balanced approach:

✅ Use AI for code assistance and idea exploration

✅ Use AI to challenge assumptions and surface alternatives

✅ Use AI to generate implementation patterns once a decision is made

❌ Don't use AI as architecture authority

❌ Don't conflate "modern" with "better for our context

❌ Don't undervalue stability and operational knowledge

When enthusiasm meets AI validation, it's easy to feel emboldened to "fix" things that aren't actually broken. The solution isn't to block change—it's to insist on evidence.

Low-Cost Experiments Over High-Risk Rewrites

If a proposed change has merit but high uncertainty, suggest bounded experiments:

• Build a proof-of-concept in a non-critical service

• Run A/B tests with controlled traffic

• Measure actual performance improvements in production

• Document what you learn about operational complexity

This protects the system while allowing innovation. It shifts the question from "Should we do this?" to "Let's validate this works for us."

Conclusion: Balance, Not Resistance

The future of software architecture will absolutely include AI-assisted decision-making. Models will get better at understanding context, constraints, and operational trade-offs. That's exciting, and we should lean into it.

But today, we need to maintain a healthy skepticism. Not because AI is bad—but because architecture decisions have consequences that unfold over years, not milliseconds. Consequences that affect on-call rotations, customer trust, and team morale.

Before we change a system that customers trust and engineers understand, let's make sure the benefit clearly outweighs the risk. Let's quantify the improvement. Let's validate against operational reality.

**Innovation isn't about adopting the newest pattern. It's about making systems better for the people who build them and the customers who depend on them.**

And sometimes, the most innovative decision is recognizing that what you have is already working—and protecting it.