Understanding what a good infrastructure strategy looks like is one thing, but executing on it in an industry that rewards speed above almost everything else is another entirely.
Across the industry, the pressure to move quickly and ship early is real and rarely lets up – and for AI teams, that tension has a specific and costly failure mode, where the consequences are rarely visible until they become very difficult to unwind.
In this instalment of our In Conversation series, we speak with Cao Hoang, Stelia’s VP of Platform Engineering, about where the “move fast” philosophy starts to work against organisations at the platform level, and why scaling AI reliably and securely demands a view of the full system – not just the model sitting on top of it.
The pressure to move fast in the industry is real – but where does that philosophy start to cost organisations more than it saves them when it comes to platform decisions?
“Move fast and break things” is a fine philosophy for UI updates, but it struggles when applied to AI platform engineering. Prototyping AI is deceptively simple right now – anyone can hook up an API to a notebook and create something impressive. The pressure to ship often leads teams to push these happy path prototypes directly into production.
The cost manifests as structural technical debt. When you optimise entirely for a short-term demo win, you skip building the non-functional requirements – automated failovers, state management, and CI/CD pipelines for machine learning. Trying to retrofit these architectural foundations under a live, high-traffic AI system is like trying to rebuilt the foundation of a building while it’s fully occupied. It risks significant downtime and usually grinds all new feature development to a halt.
Taking into consideration the extra steps required up front to build a resilient path to production actually allows developers to move faster and more safely in the long run.
Interestingly, optimising for the quick win represents a pattern that extends well beyond the development pipeline alone. As Cao explains, the same instinct to isolate and simplify – to focus on one layer of the stack in isolation rather than the complex interdependencies across all of them – creates equally significant risks when it comes to scaling for any meaningful commercial impact.
When it comes to scaling AI reliably and securely, is it genuinely a full systems challenge – and do you think the industry is treating it as one yet?
It is fundamentally a distributed systems challenge, and unfortunately, a large part of the industry is still lagging in treating it as one.
People are mesmerised by GPUs and custom silicon – Neural Processing Units, specialised cloud chips – but an AI workload is a complex interplay between compute, high-speed networking, and massive volumes of storage. If your storage can’t stream data fast enough, or your network topology drops packets, those highly expensive chips sit idle waiting for bytes.
The same applies to security. The industry is hyper-fixated on securing the model itself – worrying about prompt injection and output guardrails. Those are critical, but securing the model without securing the infrastructure beneath it is like putting a heavy bank vault door on a flimsy frame. If an attacker can bypass the model to access the unencrypted object storage where your proprietary weights and RAG data live, or if your inference servers lack network isolation, the application-level guardrails fail entirely.
True AI scale requires a holistic view across the entire stack.
The pattern Cao describes – fixating on one layer while the rest of the system goes unexamined – is one of the most common and costly mistakes teams make as they move from AI experimentation into production at scale.
Changing this demands a shift in how AI infrastructure is thought about from the outset – not as a series of discrete decisions about compute, networking, or model security, but as an interconnected system in which each layer depends on the integrity of the ones beneath it.
It is also the problem that Stelia AI OS was built to address – replacing the fragmented approach to AI infrastructure with a full-stack foundation that allows engineering teams to focus their capacity on building capabilities that can scale from the outset rather than on constantly integrating and maintaining the stack beneath them.
In the final instalment of this series, Cao will define what good actually looks like – setting out what truly production-grade AI delivers for an organisation, and what the teams already ahead are doing today to get there.
