A few years ago, when we started approaching large enterprises with Stormly, I heard the same objection repeatedly:
“You’re cloud-based. That’s going to be a problem.”
For some organizations, it genuinely looked like a dealbreaker. Their security policies, procurement processes and internal architecture had been designed around keeping sensitive systems within infrastructure they controlled.
What was interesting was what happened next.
In several conversations, cloud went from being the obstacle to being part of the reason the project could happen. Implementation was faster. There was less infrastructure for the customer to maintain. Experiments could start without a lengthy internal deployment project, and the economics were considerably easier to justify.
Organizations adapted their internal processes because the practical benefits were worth it.
Now AI is making the pendulum swing again.
AI has made data location uncomfortable again
The concern is understandable.
Enterprise AI isn't just another application running in the cloud. Companies are considering systems that can access customer behavior, transactions, internal documents and other sensitive business information, and increasingly reason across those sources.
That changes the conversation.
CIOs understandably start asking: Where is this data going? What does the model provider see? Is it retained? Could it be used for training? What can the AI access? Where does processing physically happen?
The instinctive response is often: perhaps we should bring everything back on-premise.
But I think that frames the problem incorrectly.
The question shouldn't be cloud or on-premise?
It should be:
What exactly are we trying to protect, from whom, and at what cost?
On-premise doesn't automatically mean private
There's a psychological comfort to having servers physically under your control.
But infrastructure location is only one part of security.
An on-premise AI system with overly broad permissions, poor auditability, unnecessary copies of sensitive data and badly managed credentials isn't inherently safer than a well-designed cloud system.
Likewise, putting an AI model inside your own network doesn't answer questions about what that model should be allowed to see.
We've had to think about this ourselves because Stormly works with behavioral and ecommerce data. An AI can be asked a broad question such as “Why did sales drop?”, then determine which analysis is appropriate, use an existing analytical report or generate SQL when necessary, and potentially incorporate external market trends.
For that to work, the system needs meaningful access to business data.
The important architectural question isn't simply where the server running that analysis sits. It's what data crosses each boundary, what the system can do with it, and whether those boundaries are controlled and understandable.
Cloud solved problems enterprises shouldn't casually recreate
There is also a cost to moving backwards.
Cloud adoption wasn't driven purely by fashion. It solved real operational problems.
Teams gained elastic capacity without buying infrastructure for peak demand. Software vendors could deploy improvements centrally. Companies could experiment without first committing to large hardware projects. Smaller teams could operate systems that previously required substantial infrastructure departments.
AI amplifies some of those advantages because its infrastructure requirements can be particularly demanding and fast-changing.
Moving workloads on-premise may be absolutely justified for certain data, industries or use cases. But doing it as a blanket response to AI anxiety risks recreating years of infrastructure complexity without necessarily addressing the actual privacy problem.
I've seen how quickly the equation can change when an enterprise moves from asking “Can we allow a cloud product?” to “How can we make this cloud product work safely?”
Those are very different conversations.
Think in boundaries, not locations
I think enterprise architecture is heading toward something more nuanced than another great migration either into or out of the cloud.
Different parts of an AI workflow will live in different places.
Highly sensitive data may remain inside infrastructure controlled by the organization. Specialized cloud services may process narrowly defined datasets. External AI services may receive only the context necessary for a particular task. Other workloads may remain entirely cloud-based because the security and operational trade-offs make sense.
The architecture becomes a collection of trust boundaries, rather than one big decision about location.
For CIOs evaluating AI infrastructure, I would start with four questions:
- Which data does this AI actually need? Don't give it access to an entire environment because doing so is technically convenient.
- What leaves our trust boundary? Understand not just storage location, but processing, temporary datasets and third-party services.
- What can the system do? Reading analytics data and changing a customer's account are fundamentally different permission levels.
- Can we reconstruct what happened? An AI system handling consequential business data should leave an understandable trail from the user's request to the resulting analysis or action.
Only after answering those questions does cloud versus on-premise become particularly useful.
The pendulum doesn't need to swing back
I understand why AI is making enterprises reconsider decisions they made during the cloud transition.
That's healthy.
But I don't think the lesson of the AI era will be that moving to the cloud was a mistake. Nor will every workload inevitably move back on-premise.
The more useful lesson is that where software runs is a poor substitute for understanding how data actually moves through it.
The best architecture may be cloud, on-premise or increasingly a combination of both.
A CIO's job isn't to pick a side.
It's to know where the boundaries should be, and make sure the technology respects them.