Stelia CTO Dave Hughes joined the MLOps Community meetup in Munich on 12th May, speaking to a room of engineers and practitioners at Mindspace Stachus.
In his session, Dave drew on his experience building production inference systems across multiple market verticals – from media and entertainment to agriculture – to make the case that deploying AI at scale is far more than a model problem. He argued that the model itself accounts for roughly 20% of the challenge in a production inference product, with data engineering, infrastructure, orchestration, and governance making up the rest. As he put it plainly: “garbage in equals garbage out” – and if you don’t design with data in mind from the start, the product will fail regardless of the model.
Dave also walked through the full anatomy of an inference product – from data collection and transformation, through the inference engine and API gateway, to orchestration and security – and shared where the real friction tends to appear in practice.
The evening also featured a talk from Alexej Penner, founding engineer at ZenML, on architecting durable execution for enterprise AI agents.
Watch the full talk here:
The conversation covered:
- Market-specific inference products: why the technology stack differs across market verticals such as adtech, retail, healthcare, agriculture, and legal – and how Stelia approaches each.
- The infrastructure reality: navigating the orchestration challenges of VLLM, LLMD, and Kubernetes at scale – including multi-tenancy and resource consumption issues that emerge as you scale to tens of thousands of users and beyond.
- Static vs real-time data: why most teams start with static datasets and the architectural decisions to make early so that the move to real-time data ingestion doesn’t require a full rebuild.
- Governance as a product concern: from regional compliance in adtech to brand safety in generative AI, why you must design with governance in mind.
Watch the full talk to hear David’s perspective on what it actually takes to take inference products from prototype to production – and the lessons learned along the way.
