Over the past decade, enterprises have grown comfortable with batch-oriented machine learning. Models that refresh overnight, inference jobs that run hourly, dashboards, and signals are updated once per day. These patterns defined industrial AI for years. They offered predictability, simplicity, and the luxury of time. If a pipeline ran long or a Spark job failed, someone in the operations team could simply re-run it or restart the workers.
But today’s frontier of applied AI does not operate on an hourly or daily cadence. Some of the most valuable AI systems in the world, from TikTok’s real-time recommendation engine to Klarna’s agentic customer-service workflows, they operate in milliseconds. The new competitive edge stems not from models alone, but from a company’s ability to feed those models the right data at the right time.
This shift is pushing enterprises into a new era: real-time AI. And real-time AI, as Jim Dowling explains in his conversation with Anders Arpteg, requires a foundation many organizations still lack: a feature store.
The feature store is no longer a niche technology. It is becoming the data layer that operational AI depends upon, an essential bridge between messy, fragmented enterprise data and the precise, consistent signals that production models require. Dowling, known informally as “Mr. Feature Store,” has spent nearly a decade building systems that demonstrate why the future of AI hinges on this concept. His upcoming O’Reilly book, Building Machine Learning Systems with a Feature Store, distills this experience into a mental model for building production-grade AI systems that scale from batch to real-time to agentic workflows.
Across the discussion during the Episode 170 of the weekly AI After Work (AIAW) Podcast, several themes emerge: the growing divergence between batch and real-time AI, the necessity of unifying historical and low-latency data in a single platform, and the operational sophistication required to keep modern AI systems dependable. This article explores those themes, contextualizing his insights with current research and developments across the industry.
1. The Real-Time Reckoning: Why Speed Has Become the Competitive Boundary
For years, the default AI architecture inside large enterprises was batch-based. The reasons were practical. Batch jobs are easier to orchestrate, easier to recover from, and easier to wrap guardrails around. They suit the slow-moving nature of many enterprise processes, from forecasting to financial risk scoring to next-best-offer modeling.
Batch remains the dominant mode today with estimated 70–80% of enterprise ML workloads still running in batch. But real-time AI is exploding in strategic importance for three reasons:
First, user expectations have shifted. Consumers no longer tolerate delays in personalization. TikTok and Instagram have trained them to expect instant reactions to their behavior. Recommendation systems now compete not only on relevance, but on immediacy. Dowling describes TikTok as “probably the most valuable AI system in the world today” because its real-time predictions create a continual feedback loop between user behavior and content ranking.
Second, industrial use cases increasingly require temporal precision. Fraud detection is meaningless if a model responds minutes after a transaction. Supply-chain predictions lose value if signals arrive after a logistics action has been taken. In healthcare, early warning systems depend on continuous monitoring, not hourly aggregations.
Third, agentic workflows or AI systems capable of taking action autonomously need fresh, actionable data. Large language models (LLMs) act as reasoning engines, but without access to real-time operational data, they cannot generate correct responses. A customer-service agent cannot rely solely on vector embeddings of policy documents; it must see the user’s latest transactions, events, and state.
Real-time AI, in other words, isn’t a niche category. It’s the operating mode required for any AI system that directly interacts with users, processes financial transactions, reacts to sensor streams, or orchestrates workflows. And as Dowling notes, the moment the industry became excited about LLMs, “budgets swung right over to LLMs,” slowing the expected adoption of real-time systems. But the need never went away; it simply became masked by the hype.
Now, as companies move their LLM proofs of concept into production, the referent problem reappears: LLMs for enterprise applications need structured, temporal, fast data which is the very domain where feature stores excel.
2. Why Real-Time AI Is Hard: The Data Challenge Most Enterprises Underestimate
If building a real-time AI system were simply a matter of plugging Kafka into a model, every enterprise would already be doing it. The difficulty lies in reconciling three properties that traditional data architectures rarely satisfy simultaneously:
Volume, variety, and velocity. Real-time systems need low-latency access to fresh data (velocity), but also access to large, historical datasets (volume) for model training. And they must unify multiple, often inconsistent data sources (variety) into structured features.
Dowling points to Uber’s Michelangelo platform as the earliest major production example of this architecture. Michelangelo combined:
• a columnar store (for massive amounts of historical training data)
• a row-oriented, low-latency store (for real-time context and online features)
Uber coined the term feature store to describe this unified layer for preparing, storing, and serving features consistently across training and inference. That concept now underpins modern operational AI.
But most enterprises still attempt to assemble this architecture themselves, piecing together Spark jobs, data warehouses, streaming jobs, and various cache layers. As Dowling notes, “people don’t write if-then rules anymore.” They rely on models, but those models can only be as good as the feature pipelines feeding them.
This challenge is made visible when companies attempt to build streaming AI use cases on top of warehouse-first or lakehouse-first stacks. If your data architecture is designed for batch, you cannot retrofit it into a real-time system. Databricks’ micro-batch approach, Snowflake’s second-scale response times, and many lakehouse storage formats reveal limitations when low-latency joins and millisecond-level consistency are required.
This is the root of the divergence emerging across the industry: batch-first platforms that excel in data analytics versus real-time-first platforms engineered for operational AI.
The Birth of the Feature Store: A Data Layer for Consistency, Governance, and Speed
A feature store solves three intertwined problems that real-time AI systems face: consistency, correctness, and latency.
1. Consistency Between Training and Inference
In classic ML, one anxiety has haunted teams for years: Is the data used during training the same data generated during inference? Inconsistency, even subtle schema drift or transformation mismatch, leads to unpredictable model behavior. A feature store enforces consistent transformation logic across batch pipelines, streaming pipelines, and online inference services.
2. Correctness and Temporal Alignment
Real-time AI must obey a fundamental constraint: no future data may leak into the model. But aligning timestamps across multiple sources is fiendishly hard. As Dowling explains through his air-quality example, the system must perform an as-of join:
“If I get this air-quality observation, what was the weather at exactly that time?”
Most databases do not support temporal joins. A feature store that implements point-in-time correctness solves this for both training and inference. In domains like fraud detection, this is literally the difference between a usable model and a misleading one.
3. Low-Latency Feature Serving
Real-time models must access features within strict latency budgets – often 10–50 milliseconds end-to-end. For LLM-powered agents that perform multiple tool calls, each millisecond matters.
This is where online feature stores shine: engineered, low-latency databases optimized for fast key-value lookups and high concurrency. Some platforms, like Hopsworks, use an in-memory distributed database to achieve these real-time constraints; others integrate with cloud-native key-value stores like DynamoDB. The implementation differs, but the requirement does not: AI systems need features fast.
4. When Real-Time Goes Wrong: The Cloudflare Incident
One of the most striking examples Dowling discusses is Cloudflare’s 18-hour outage, triggered by a potential internal feature-management system that malfunctioned. The company might have rolled its own feature store on top of thier warehouse, and a faulty query introduced duplicate rows into feature files, causing cascading failures across edge nodes. In Dowling’s retelling, the problem was simple but catastrophic: the query “wasn’t tested,” leading to data consistency issues that took the system down.
This incident underscores a critical truth about real-time AI systems: feature pipelines are software, and software must be tested, versioned, monitored, and governed. The failure modes are subtle – not missing data, but incorrect data; not broken pipelines, but semantically wrong joins; not unavailability, but time drift.
The Cloudflare case is a warning for enterprises still assembling their own feature infrastructure from generic components. Real-time AI raises the cost of subtle errors. A single incorrect feature can cause a model to misclassify thousands of events per second, triggering financial losses, security gaps, or cascading operational failures.
The lesson is not that companies should never build; it is that feature engineering infrastructure carries similar complexity to database engineering, and must be treated with similar rigor. This is why the feature store ecosystem has consolidated around a handful of mature platforms, including Hopsworks while dozens of earlier entrants have disappeared. The complexity of the problem rewards long-term engineering investment.
5. A Mental Model for Building Operational AI Systems
Dowling’s most enduring contribution may not be a platform, but a conceptual framework. In both his book and his university teaching, he divides AI systems into three pipelines:
- Feature pipelines that are transforming raw data into structured signals
- Training pipelines converting features into models and
- Inference pipelines which are applying models to new data in real time or batch
While this decomposition seems intuitive, its power lies in how it aligns teams and interfaces. Data engineers own feature pipelines; ML engineers own training logic; application teams own interfaces to inference. And the feature store acts as the contract between these stages. A clear, governed, schema-defined layer that prevents “spaghetti pipelines” from forming within organizations.
This model supports batch systems (like Spotify’s weekly recommendation updates), real-time systems (like Zalando’s product ranking), and increasingly, agentic LLM systems, which behave like real-time AI but with richer reasoning stages. As Dowling puts it:
“Agents can actually have an interface, a schema, an API.”
This is a subtle but profound shift. LLM agents do not simply consume strings; they consume a structured world state. When a customer contacts a service agent, the LLM needs vector-search results (policies), tabular features (transaction history), and streaming events (recent actions). A generalized feature store becomes the unified substrate on which these agents operate.
6. Agentic Workflows and the Coming Data Renaissance
LLM-powered agents are rapidly becoming central to enterprise transformation agendas. But they also expose deep data limitations. In many cases, organizations know they have the right data but cannot get it into the hands of the agent at the right time or in the right format.
Vector databases are not the answer to everything. They excel at semantic retrieval over unstructured content, but they are not suited for transactional or temporal data. Real agents need both:
- Semantic context (policies, manuals, documentation)
- Operational context (user state, transactions, events)
- Temporal context (sequence of interactions)
As Dowling notes, “some of that private data will not end up in vector databases and probably will never be there.” Instead, it must come from operational data stores, real-time streams, and the feature store.
This is exactly where feature stores will evolve over the next three years: from an ML-specific abstraction into a general-purpose context engine for agents. The distinction between offline features, online features, context windows, and RAG embeddings will blur. Enterprises will need a unified layer that can:
- Store and serve structured features
- Manage temporal joins
- Surface real-time context
- Feed multimodal inputs into agents
- Track lineage, versions, and quality metrics
The feature store is on track to become this layer.
7. Why Europe’s AI Ambitions Depend on Real-Time Infrastructure
Dowling also touches on digital sovereignty and Europe’s position in the global AI race. Europe does not need to outspend the US or China on trillion-parameter model training runs. It can instead excel in operationalizing AI especially industry-specific, domain-rich, privacy-conscious systems that depend heavily on structured and temporal data.
Enterprises that master real-time AI and data infrastructure will outperform those that simply consume generic LLM APIs. European companies have deep domain knowledge, rich operational data, and strong regulatory environments all ingredients for differentiated AI systems. The missing piece is the infrastructure to convert this data into real-time intelligence.
If Europe focuses on adaptation, fine-tuning, and application-layer excellence, supported by platforms that provide secure, on-premise or sovereign-cloud data infrastructure (including systems like Hopsworks), it can compete in areas where model size matters far less than model grounding.
Real-time AI is the strategic layer where Europe can lead.
8. Lessons for Leaders: Building AI Systems That Actually Work
Based on Dowling’s insights and broader industry trends, several lessons emerge for senior data and AI leaders:
1. Real-time AI isn’t optional; it’s situationally mandatory – Any use case requiring immediate response like recommendations, fraud detection, support automation, anomaly detection must operate in low-latency mode.
2. Accelerate by simplifying the mental model – Teams adopt AI faster when they share a common architecture: feature pipelines, training pipelines, inference pipelines, connected via a feature store.
3. Batch-first stacks cannot magically become real-time stacks – If your primary data infrastructure is optimized for analytics, you will hit latency ceilings quickly.
4. Testing and observability become existential in real-time systems -Every component must be versioned, every transformation transparent, every join reproducible.
5. Agentic systems demand a unified data layer – LLMs are powerful, but without real-time structured context, they hallucinate, misinterpret state, or recommend the wrong actions.
6. You can’t build safe, reliable, low-latency AI without a feature store- You can assemble workarounds, but the operational burden only grows as systems scale.
9. The Road Ahead: From Feature Stores to Real-Time AI Platforms
As agentic workflows mature and real-time AI becomes mainstream, the feature store will evolve into something broader: a real-time data platform for AI.
The next generation of feature stores will likely include:
- Tight integration with vector search for hybrid retrieval
- Native point-in-time semantics for multimodal data
- Built-in support for online fine-tuning and continual learning
- Real-time monitoring of concept drift, data drift, and model performance
- Sandbox environments for testing agentic workflows against historical event streams
- Automated lineage for every feature, join, and pipeline
This evolution is already underway, visible in both open-source communities and commercial platforms like Hopsworks that are pushing the boundaries of real-time feature engineering.
What’s clear is that the feature store will not remain a niche operational tool. It will become a central, enterprise-wide component of the AI stack and it will be as foundational as the data warehouse became for analytics.
Real-time AI is not simply “faster ML.” It represents a different paradigm of intelligence: one where systems engage with users, processes, and the physical world in the moment, informed by fresh data, consistent features, and structured context. This is the direction in which the most valuable AI systems in the world are moving. And it is the direction enterprises must move if they want their AI investments to deliver outsized value.
Conclusion: Building the Foundations for the AI Systems of Tomorrow
Jim Dowling’s work, spanning Hopsworks, university teaching, community leadership, and now his new O’Reilly book offers something the AI industry rarely provides: clarity. His mental model simplifies the complexity of building production AI systems. His insistence on temporal correctness exposes the hidden pitfalls that derail real-time ML initiatives. And his perspective on agents hints at how foundational feature stores will become in the next era of enterprise AI.
As AI systems evolve from batch models to real-time intelligence to agentic workflows, one thing becomes clear: the future of AI is fundamentally a data problem, not a model problem. And solving that problem requires a robust foundation and a unified feature store capable of delivering clean, consistent, contextual data at the speed of modern AI.
Enterprises that recognize this early will build systems that not only work but also scale. Enterprises that ignore it will find that no amount of model sophistication can compensate for unpredictable, stale, or inconsistent data.
Real-time AI isn’t the future. It’s the new present. And the feature store is how we get there.
This article was enhanced with the help of AI tools, drawing on the podcast transcript and complementary online research. To go deeper into the source material, listen to the full episode and make your own learnings.
