The technical parameters defining enterprise data infrastructure are being entirely rewritten. As modern organizations demand rapid continuous integration for data schemas, decentralized architecture models, and safe ways to navigate sensitive production repositories, traditional transactional database systems frequently become costly operational bottlenecks. The immediate priority for engineering teams is no longer just expanding raw storage capacity, but optimizing data layouts for extreme execution speeds and eliminating pipeline delays.
Achieving this layer of operational agility requires shifting away from rigid, monolithic architecture designs. To build dependable backend setups that can scale under modern workload strain, engineering teams must establish self-service data ownership frameworks, align database updates with automated application pipelines, and deploy advanced privacy-preserving synthetic data flows.
Reviewing the data management frameworks shared at the Data Innovation Summit (DIS) 2025 surfaces an actionable roadmap for engineering leaders. The following five case studies examine how forward-thinking technical teams are overhauling their backend data systems to achieve maximum throughput, robust compliance, and unmatched processing speeds.
1. Reimagining Semi-Structured Storage for True Columnar JSON Performance
JSON has become the lingua franca for handling semi-structured and unstructured data in modern systems, powering everything from logging and observability to real-time streaming, mobile app storage, and machine learning pipelines. Explaining this storage breakthrough at DIS25, ClickHouse detailed how they completely reimagined JSON storage on top of column-oriented database designs. While its inherent flexibility makes it the default choice for capturing and transmitting data across distributed environments, traditional database systems routinely struggle to index dynamic paths efficiently.
The session detailed how to achieve true columnar JSON storage that is even more compact than compressed files on disk. The presentation explained why this tailored approach is thousands of times faster than traditional JSON data stores, delivering an elite user experience and unlocking performance gains never seen before. By natively supporting dynamically changing JSON paths without forcing unification into a least common type, ClickHouse provided engineers with a high-performance blueprint to implement columnar storage the right way for semi-structured data.
2. Shifting to Data-Centric Architecture Through Enterprise Data Products
Traditional data management practices frequently trap useful information inside system-centric silos, making it difficult for non-technical teams to access or use efficiently. Addressing this infrastructure barrier at DIS, TDC Net explored the transformative power of enterprise data products within a Data Mesh architecture. To eliminate standard delays associated with data retrieval, organizations must intentionally transition from system-centric thinking to a data-centric approach that aligns directly with core business needs.
The session demonstrated how to logically group data entities into enterprise data products, enabling users to request information in meaningful business terms. The presentation detailed the principles of creating a self-service data environment where business users can pick, mix, and choose their data products effortlessly in a modular, Lego-like fashion to foster innovation and data-driven decision-making. By implementing strict data governance and quality controls directly onto these enterprise data products, TDC Net highlighted how to free business data product owners from the burden of managing compliance manually.
3. Boosting Delivery Speed Through Database DevOps and Change Management
Increasing the speed of delivery for database changes has become a top driver for introducing modern DevOps practices straight to the database layer. Addressing this deployment lifecycle friction at DIS, Redgate explored the most impactful ways technical teams can bring true agility to the development and deployment of database changes. When database updates remain slow and isolated from automated software deployment cycles, organizations face severe bottleneck constraints that delay features and stall engineering velocity.
The presentation brought real-life examples to the fore, demonstrating exactly how engineering teams can transform database change management to deliver value quicker and stay ahead of competitors. Redgate experts shared actionable insights designed to seamlessly bring speed and flexibility to the entire update pipeline. By applying these practical agile transformation workflows directly to core database pipelines, the session provided data professionals with a functional path to accelerate release frequency while ensuring absolute system stability.
4. Democratizing Proprietary Repositories Safely via Synthetic Data and Agentic Analytics
A striking 95% of proprietary enterprise data remains completely untapped, and 95% of internal talent still cannot access it due to strict security barriers and technical constraints. Addressing this systemic infrastructure freeze at DIS, MOSTLY AI & Swiss Post demonstrated how they deliver on the promise of wide-scale data access and deep data insights for everyone. When sensitive consumer data remains locked behind rigid privacy protocols, engineering and data science teams face major development delays, blocking the internal innovation required to build competitive digital systems.
The session walked through how to unlock sensitive proprietary data with privacy-preserving synthetic data using their open-source Synthetic Data SDK, enabling scalable, compliant data access across diverse corporate teams. The presentation then detailed how to bridge the enterprise skill gap using the MOSTLY AI Data Intelligence Platform, where a Python-native AI Assistant empowers anyone to run agentic analytics and data science on synthetic, mock, or real data using basic natural language queries. By explaining why unlocking proprietary data safely is the next frontier of artificial intelligence and a major competitive advantage, the session provided a powerful strategy for enterprises looking to get there first.
5. Transitioning to Embedded Machine Learning Within High-Maturity Public Systems
The Norwegian Tax Authority (Skatteetaten) operates with one of the highest digital maturity levels among public organizations in Europe, collecting and administering some of the largest datasets available from individuals, firms, public institutions, agencies, and registries. Addressing this transition at DIS, Skatteetaten (The Norwegian Tax Authority) detailed how they put their data strategy into practice after explicitly incorporating machine learning and artificial intelligence into their strategic ambitions. Scaling models beyond isolated pilots requires embarking on a massive corporate transformational journey.
The session focused on their approach for building and implementing large-scale machine learning models in a cross-disciplinary, complex, and fragmented setting, bringing real-world examples from the agency’s risk scoring models for companies. The presentation shed light on common organizational, technical, and capability challenges for developing AI-enabled processes in Nordic public organizations, providing resolutions to unlock value at scale. Skatteetaten proved that data preparation is the absolute priority, demonstrating that value is 80% in the data and 20% in the models. To execute this safely, they highlighted the need to put together a cross-sectional team representing all needed expertise in the entire value chain while creating a technical environment where as much as possible can be reused transversally across the organization.
Moving Past the Architecture Bottleneck
Optimizing your underlying data infrastructure is the highest-leverage engineering play an enterprise can make. Shiny front-end applications and massive models are completely dependent on the storage layers, deployment pipelines, and access paradigms running beneath them. True scalability is unlocked only when your databases match the agility of your product engineering teams.
We are actively synthesizing these infrastructure case studies into practical engineering manuals. Stay tuned to our channel as we dive deeper into upcoming technical publications, focusing on everything from the granular optimization of specialized relational engines to decentralized governance frameworks built for complex global organizations.
