Enterprise AI, like any other technology product, must provide value. And not just across one department or one project but the entire organization. Democratization of data and AI is the ultimate goal enabling practitioners across functions to innovate and maximize the value of the organization’s advanced analytics capabilities.
According to Gartner, the goal behind data democratization is to “allow non-specialists to be able to gather and analyze data without requiring outside help”. Organizations must equip all employees and stakeholders with easy access to data and no code AI products without extensive or expensive training to achieve this goal. That is why data democratization is often referred to as citizen access.
Many findings prove that organizations that apply this approach make better strategic decisions, have higher efficiency, improved customer satisfaction, and generate more profits.
Some sources define data democratization as a trend that is part of a larger trend of technological democratization. This means that the benefits of the technology and the technological advances can be accessed by non-technical or non-traditional users, too.
Why should organizations know about the importance of the decentralized value creation of AI? And what should they do if they decide to apply a similar approach? This article will point out several steps organizations should be aware of from the data management perspective. And we will use several use cases by Telia, EQT, Pandora and other companies to emphasize those steps.
Limitations and Solutions around Data Democratization
The key factors that can foster an organization’s data democratization are machine learning, data access, deployment, culture, technology and governance. But, like any other, the process of data democratization has challenges, linked to silos and bottlenecks, complex infrastructure, data access and quality issues, and lack of collaboration and resources. Let’s explore the different use cases from the organizations mentioned above to see what they have done to overcome these challenges. Few of these use cases address the importance of decentralizing data ownership, data teams and infrastructure towards the organization’s data democratization.
Overcoming the silos and bottlenecks
The global jewellery company Pandora has around 27,000 employees, with 6,800 sales points in nearly 100 countries, and 670+ million visits to offline and online stores. The company wanted to use all data from these sources to run the company better, and make strategic and tactical decisions. To overcome any silos and bottlenecks with the centralized structure, the company decided to decentralize the team and to originate them around the tools. The company is using three different processing tools (in-memory storage and compute, data warehouse and data science workspace) to address requirements that are coming from the three main personas that generate data in the company: the enterprise analyst, the front-end engineer, and the data scientist. Other personas participating in the process of generating data include data engineers, analysts, and ML engineers.
“We started with demands of data, and then we started centralizing the data teams. At one point, it became a bottleneck, and now we are embarking on a journey to see if decentralizing the data team and if we do it properly will work for us.”, explains Febiyan Rachman, Senior Data Engineer at Pandora Jewelry.
Promote collaboration
In Pandora’s case, the company decided to partly decentralize the architecture so that the data domain teams could coordinate and collaborate better. Febiyan Rachman explains that every individual domain team will have its automated workspace, but the data product will be put in a common space accessible by the shared workspace where it is expected that collaboration will happen. With this plan, the company also envisions ensuring that the architecture is repeatable.
Overcoming complex infrastructure, data access and data quality issues
Telia is a Swedish multinational telecommunications company and mobile network operator that serves over 25 million customers in 6 different markets. On an average day the company runs over 3 million queries. According to the presentation by Karolina Perzon, Head of Governance & Strategy at this edition of the Data 2030 Summit, Telia has around 350 analytical experts, over 5,000 BI users and works with large volumes of data. Besides a massive data legacy, the company faced a complex IT source structure impacting the overall data quality.
“Within the analytics team we have been working for a long time to mitigate these issues. We have created workarounds to improve the data quality but that also became very complex and time consuming and required a lot of resources.”, adds Karolina Perzon, Head of Governance & Strategy at Telia.
The solution for overcoming the problem was an updated data strategy and a pragmatic breakdown of activities to quickly prove value of the strategy.
Reasonable usage of tools and resources
Telia also felt that it had an endless loop of proof of concepts (POCs), evaluations, and talks about what is the perfect tool and technology to move forward with data democratization. At this time, the company had implemented a catalog but the capability itself was sparsely utilized.
“It was not because of the catalogue itself, the tool or the technology, but we didn’t have a process that describes how it should be used and implemented. We also didn’t have processes for other issues, such as how to drive development, issue resolution etc.,” says Karolina Perzon. She adds that to overcome the issue, there was a request for an update of the data strategy.
Juts on the above mentioned use cases, some common proposed steps to overcome any limitations with data democratization include:
- Inventory of data and know where it is stored, which tools and technologies are used to capture, store, and analyze data, do you have resources to provide analysis.
- Identification how much data-literate the employees are, and for this there are different methods, some simple and some more complex.
- Investment in proper training and ongoing education based on the findings and the needs the organization has. Ongoing training and check-ins are vital for the data democratization journey if the organization wants to see a strong return on investment and benefit from it.
- Assessing potential data solutions. Consider budget, customer service reputation, scalability, and market prominence of each potential tool.
- Shift from centralized to decentralized data ownership/structure/teams/analytics as part of the data democratization journey.
Decentralization is Crucial for Data Democratization – Use Cases
In the following paragraphs, we’ll focus on the last step mentioned in the section above: data decentralization. Since, according to many, it is seen as an essential step towards data democratization, we’ll share a few approaches to help organizations shift from centralized to decentralized data, ownership, team and analytics. We’ll use the same companies as a positive example with their initiatives and strategies.
Setting goal and ambition
Establishing a goal, ambition and approach that is adopted to a specific organization, and that will bring value to the organization the fastest is the first step towards data decentralization.
“Taking too much of the best practice approach when it comes to establishing your data strategy, can easily become overwhelming. It requires a lot of resources, effort, and costs, and most likely, you won’t necessarily see long lasting results. I’ve seen this a number of times before. You set up a big project, you bring resources, you point to data owners, you document the processes, you might even procure tools but at the end as the project dissolves, you will lose commitment and go back to the old ways. I won’t say you should not set up a project or bring resources, what I’m saying is, there is no best practice approach that will work for every company. We need to have a very pragmatic approach in terms of how we implement this.” Karolina Perzon, Head of Governance & Strategy, Telia.
When Telia set the ambition they wanted and expected: data self-service for the company’s technical users and later scaled to the business users; data as an asset which for the company was meaning prioritizing the most valuable assets; data driven business decisions based on high quality data; and data democratization by enabling a common language cross markets and departments which, again, means that any given user within Telia, independent of their technical know-how to be able to work with data comfortably.
Optimizing for adoption of decentralization
Data projects may fail because their assessment takes much time so by the time organizations are done with the assessment, the relevance may pass.
“To overcome this our strategy is to look at the broken processes and super frustrated users within the organization… So, we find them and tell them that there is technology available to help them out with automation of repetitive tasks. With other words we tell them to iterate and don’t be afraid of shortcuts. Like that you can fail fast and learn from the mistakes and find good solutions. The next level is – solve users’ problems. This leads to strong allies and endorsement in the data journey. Later on we can have access to this type of data which is important for the auditors.”, explains Pedram Birounvand, Head of Data Management at EQT Group.
Building data management strategy and structure
When Telia started shaping their data strategy these are the five key areas that they started to look at: technical foundation as an enabler, processes and organization, common language as a factor to unlock greater power, communication and training, and privacy and security incorporated by design.
Data governance enablers
Data governance helps to know what data organizations have, who is the consumer, and what control the organization has on different data points.
“So, next time when a team decides to build something, go to a data catalog and see to reuse a metric or build a metric from scratch. While you do the documentation you start to find other types of synergies that you can start using like dynamic data masking (important when working with GDPR) and specifying object tagging (important when working with Personal Identifying Information – PII data and confidential and sensitive information).”, explains Pedram Birounvand, Head of Data Management at EQT Group.
Having technology and agile data infrastructure
Being iterative as part of the organization strategy requires having good technology, think the data experts from EQT Group.
“Compared to the previous traditional data architecture with a centralized and normalized database, we think about different slices we want to do with the data and we are building data teams surrounding those different data products. Those are small teams of 3-5 people focused on certain data domains and they build their own. This has been proven to be agile and successful and they know their customers and understand their pains. But like every architecture model this one contains errors as well.”, explains Pedram Birounvand, Head of Data Management at EQT Group.
Organizing and structuring the data team
As the data demand grows and there are more use cases executed, organizations may experience bottlenecks. To tackle the challenge, they can think about decentralizing the data teams. As in the Pandora case.
“So, we have an ingestion team that takes care of ingesting data from hundreds of different sources, that puts the data into a data lake and needs to provide downstream teams with easy to organize raw data. We also have a data warehouse team that has the purpose of modeling the data and making it standardized and ready usable by the whole enterprise. We also have an in-memory analytics team that takes the data from those two platforms and puts it in the in-memory store for the enterprise analysts to use. These four teams are working closely together. Besides them we have a front-end engineering team and the enterprise dashboard portal team. The platform team works to support the initial four teams. It was not easy to come to this structure, it was not an easy journey to get there.”, elaborates Febiyan Rachman, Senior Data Engineer at Pandora Jewelry.
Data management culture and value creation across functions
Building data management culture requires organizations to work close to the business in regards to different types of objectives including: accountability, data quality and sense of urgency. And regarding the communication with the language of value and not technology, Jo Coutuer, Founder of Datamerit and former Chief Data Officer at BNP Paribas Fortis says:
“Anyone in the position of CDO or running data initiative should be the agent of value discovery and agree with the finance people about the financial mechanisms or with the peer about how you are going to recognize it, on which accounts it can be booked…speak with the language as the finance people, chief revenue office or anyone else responsible for the revenue recognition would speak.”
Key Takeaways
There is no silver bullet and best use case to follow when leading an organization towards data democratization. Define your own best practice. See what works best for your organization. There is no one product, one process, and one approach that will get you to what you want from data democratization. But a combination of everything may be the right fit for organizations. The real value is exchanging experiences on data management strategies, governance, teams and how to decentralize them, and how to decentralize the ownership of the data. For more best practices on this, we recommend exploring the agenda of the Data Innovation Summit for 2023. Also, explore Hyperight Premium content to hear first-hand stories from the best global leaders in data democratization.
Featured image: Tyler Franta on Unsplash
Add comment