The number of organizations with a data-driven mindset or strategies in their everyday operations is growing. At the same time, the amount of data that is available for the organizations is also increased. However, there are some challenges organizations face, like the challenges regarding access to data or complexity of the environment to work in. How is data management evolving? It is crucial for organizations to follow these trends that can help overcome the obstacles during their data-driven journey.
Hyperight organized several events (Data Innovation Summit in the Nordics and in MEA region, Data 2030 Summit, NPA Summit, NDSML Summit etc.), and the team had a chance to talk with the leading data management practitioners that had presentations at these events on the most trending topics. In this article, we are summarizing some of their views on what trends in data management can be expected in the upcoming period.
Data Mesh as a Concept and Architecture
Data Mesh was one of the biggest data management topics discussed this year and will be a trend in data management during the next few years too, according to Mahmoud Yassin, Lead Data Architect at ABN AMRO.
“Data Mesh challenges the traditional/classical thinking of how to store and manage data in a central place called data warehouse or data lake. It is introducing a denormalized approach of storing, processing and sharing data across what is called ‘Domains’ and dealing with data as a ‘Product’ while keeping a ‘federated governance’ layer on top.”, says Mahmoud Yassin.
Data Strategies Based Architectures are Here to Stay
In an interview with Håkan Jonsson, Lead Architect Data Strategy at Swedbank, shared that data strategies are here to stay in the next 1 – 2 years as one of the core strategies of the organizations.
“…In our Bank, data strategy is one of our core strategies, we have a set of those among the strategies for the private consumers, general IT and so on. I generally think that the role of the architecture is to enable whatever the strategy directs.”, says Håkan Jonsson.
Listen to his presentation at the 6th edition of the Data 2030 Summit on “Swedbank’s data strategy-based architecture for increased delivery – balancing legacy and new directions”.
Modern Data Stack and Data Fabric
In an interview with Matt Turner, Director, Partner and Industry Marketing at Alation we asked about the trends that are impacting data strategies and that are important for the future of data management. Beaside data mesh, he pointed out modern data stack and data fabric as crucial trends. This is what he said:
“The modern data stack is about creating very fast moving pipelines and data stacks to bring data for specific uses, and to bring these together in the cloud. This allows them to be created very quickly and speeds time to decision. This approach breaks down data silos and monoliths with agile pipelines. Key to modern data stack success is data intelligence. Information about all your data speeds the creation of data pipelines and empowers everyone using the data to know where the data came from and how to use it. In addition, a megatrend that is a bit older but still important is data fabric. Data Fabric is a design pattern to help you architect your systems to deliver data mesh and the modern data stack. We see all of these trends coming together to tackle the IT bottleneck and provide new ways of thinking about the balance between centralization and decentralization.”
Listen to his presentation about these three mega trends in data management that he delivered at the 6th edition of the Data 2030 Summit.
Metadata Becomes More Important
Without metadata, data itself can be useless. Metadata will become more important in the future and will be key to the success of any organization’s data journey. This is what Mahmoud Yassin, Senior Data Manager at Booking.com in an interview for Hyperight.com.
“Metadata is being generated at any life cycle of your data journey. Many organisations just don’t have a vision on the great value of gathering and maintaining all types of metadata and how this metadata can dramatically improve the data strategy and the execution of being more data driven. That’s why metadata is sort of neglected while harvesting and using it can add big value.”, added Mahmoud Yassin.
More about the metadata, its definition, types of metadata and ways to utilize it, find out in his presentation “Role of {Active} Metadata {Lake} in any successful data architecture”.
Data Catalogs Embedded in the Day-to-Day Products Suite
Data Catalog will have a vital role in the next few years. It will be seen as a powerful approach to IT and business teams reconciliation, and will be embedded in the day to day products suite.
“For now we still spend a lot of human time in documenting the data catalog and this is where the technology should support. Human intervention should only be positioned to bring operational knowledge and expertise to the data catalog. Everything that could be automated should be automated. With more cloud based and self-service solutions the data centricity will become a very important skill in organizations so data literacy will also be part of these courses in organizations. We also have the concept of data mesh that allows domain data drive architecture. A lot of businesses will focus on their domain across global data catalogs. Last but not least, data catalogs will be part of the people’s productivity tools. We now use emails, chats etc so data catalogs will be embedded in these day to day products suite to allow data catalog content to be surfaced to users without having to connect with specific applications.”, explains Laurent Dresse, Chief Evangelist at DataGalaxy.
Listen to the interview with him where he is sharing more about the importance of data catalog.
Data Platforms and SLA/SLOs for Data Products
More about the future of the data platform, in the interview with our team Iker Martinez de Apellaniz, Adevinta shared:
“I’ve started to observe Data mesh platforms coming into the market and Data platforms changing their approach to support Data mesh. I see this as a risk because Data mesh starts with a change of mind and culture, and trying to buy Data mesh is not going to work well. But on the more positive side, I think using SLA/SLOs for Data products is going to grow, and the complexity there will be how quality tools like Great Expectations or Deequ can work in these types of platforms and how they can work as a trigger of alarms that cascade FROM the producing teams TO the consumers, and not in the opposite direction, as it is happening currently.”
User-Centric Data Platforms
“Data platforms have two possible directions: data mesh or centralized Data Platform in terms of operating model. Technologies are all developing very fast and it’s getting more important to have internal competences to lead the architecture. Agile ways of working are also taking a bigger role in both advanced technologies as well as traditional IT systems. The collaboration model and clear data strategy is still missing in many companies.”, said Kristiina Tiilas, Head of digital platform at Outokumpu Oy.
Listen to her full presentation on how to make a data platform compelling to business users.
Sustainable Value Management for Data and AI Transformation
“It is my prediction that a lot of organizations will go through Gartner’s infamous “trough of disillusionment” when it comes to data and AI. Especially the ones that have overhyped data and AI without having put in place value monitoring and rigorous value based portfolio management. A second trend is that investors and financial markets will start to inquire actively about the expected returns from AI. The value of an AI portfolio will become a factor in or even a proxy for the public and private valuation of companies. So company stakeholders should stay ahead of both the disillusionment as well as the investors by professionalizing their own data and AI value management. After all, nobody can express and appreciate the stakeholder value better than the company’s own.”, elaborates Jo Coutuer, Founder of Datamerit and former Chief Data Officer at BNP Paribas Fortis.
Read the interview Hyperight.com team had with him, or listen to his presentation on “Why rigorous value management is key to sustainable data and AI transformations”.
Data Strategies for Trustworthy AI
Having data strategy and aligning it with organization’s business strategy is important to drive value out of data, but also to make sure that all the data is secure and trustworthy.
“It’s important for companies to approach AI modeling strategically, and with humans at the center who are likely to embed governance and trust from the start. Companies without any AI life cycle governance – what we sometimes call “level 0” – are typically where a company starts its AI journey. While this approach provides a lot of flexibility, it can introduce significant risks that are nearly impossible to even evaluate. Potential pitfalls of this approach include steep regulatory fines, knocks against corporate reputation, or even accusations of bias.”, says Seth Dobrin, Global Chief AI Officer at IBM.
According to him, the time for companies to act is now, and adds:
“The study data suggests that those organizations who implement a broad AI ethics strategy interwoven throughout business units may have a competitive advantage moving forward. The study provides recommended actions for business leaders and is a great resource for organizations looking to embed AI ethics into their practices.” adds Seth Dobrin.
If you want to know more about what these experts forecast, we advise you to read interviews with them, watch their talks by signing in for Hyperight Premium, or be part of some of the upcoming events where you will have a chance to listen to presentations by some of these names and companies’ representatives. And, if you have insights to share about the future of data management, get in touch with us via [email protected].
Featured image: AltumCode on Unsplash
Add comment