A few years ago, predictions for 2020 saw optimistic advancements in data analytics fully aligned with business needs and goals. However, today’s reality is much different, where companies are experiencing hurdles in their data pipelines preventing them from successfully running data analytics, or integrating analytics at all, and getting actionable insights and business value. There is a new concept in the data analytics market that introduces the vital continuous flow of intelligence and action, based on real-time, up-to-date information and is designed to compel, to trigger action and engagements with the insights that it generated – active intelligence.
The topic of active intelligence was the reason for our encounter with Dan Sommer, Senior Director, Global Market Intelligence Lead at Qlik, where we discussed the idea behind active intelligence and the ongoing paradigm shift happening on the BI and data analytics scene.
Hyperight: Hi Dan, it’s a pleasure to talk to you for the third time around. The last time we met, we discussed how the COVID-19 crisis was affecting the BI and Analytics landscape. Today, we are discussing getting real business value out of data, which, unfortunately, few companies manage to do. Even in the era of AI industrialisation, companies face blockages in their data pipelines to go from raw data to insight to action. Why do you think this is?
Dan Sommer: Thank you. I see three key reasons why we’ve seen both blockages and leakages in the data pipeline. The first reason is that we haven’t had the same agility on the data management side, as we’ve had for analytics. This means really long cycles, out of step with the actual business requirements, leading to many business users either not using data at all, or just getting their own. This can now be remedied through better technologies, in combination with an agile methodology called DataOps.
The second reason is that there haven’t been enough integration points between collecting data, integrating it, moving it to the right place, and then analysing and actioning it. This has all been done by siloed technologies. I think the consolidation in the market between data management and analytics will do a huge amount here to “fix” the pipeline, as we’ll see more integration between the different parts. Thirdly, I think we’ve had an issue at the data model and metadata level. We see a lot of exciting innovation here where data producers and information consumers can collaborate earlier, through better data virtualisation, data catalogues, and improved business logic. That will act as connective tissue.
Hyperight: I recently came across a new concept in BI – Active Intelligence. Could you tell us more about what active intelligence represents, and how it differs from traditional, passive BI?
Dan Sommer: Once you have that end-to-end value chain in place, you have the opportunity to do business at the speed of data. Then, a big opportunity arises in moving BI from passive, to active. Traditional BI – passive BI – is based on pre-configured, curated sets of data, and designed to inform but not necessarily compel action. Active Intelligence, in contrast, is based on a continuous flow of intelligence and action, based on real-time, up-to-date information and is designed to compel, to trigger action and engagements with the insights that it generated. By bringing data integration and analytics capabilities together, we can exploit new use cases, where we bring together data at rest with data in motion, and other derivative data, to solve wholly unique and complex business problems. We can enable much more real-time and operational monitoring of what’s happening within the business, and as a result, trigger action and trigger events.
Hyperight: Many organisations invest heavily in building data pipelines and establishing data analytics, but fail at the last step of putting data into the hands of the people that can benefit most from it. What would you advise companies to do in order to see their data put in the right use?
Dan Sommer: Adoption and gleaning actual insights and action is the hardest part. From a technology perspective, you have to flip the paradigm, where analytics come to you, rather than you have to come to analytics. This means building much more analytics into our workflows and processes. Active intelligence is a more sophisticated state of embedded analytics, that enables insight delivery, data dialogue and action. This will put data insights into the hands of not just information workers, but also “workers with information”. Of course, data literacy needs to be an ongoing effort to raise people’s data skills through-out.
Hyperight: There is a new paradigm shift in data analytics that moves away from passive analytics to active analytics. Part of this paradigm shift is the forward-looking vision of augmented analytics. Could you share more about this vision for augmented analytics?
Dan Sommer: Augmentation, i.e. AI helping humans do analysis better, needs to be part of the whole information value chain that enables active intelligence. For example, for decades, we’ve spent 80% of our time doing data prep, and only 20% doing analysis. That ratio hasn’t changed. What if we could flip that proportion, through AI doing more of the hard and tedious tasks? I think we increasingly also will use augmentation to move from self-service to self-sufficiency. What I mean by that, is that in this world of active intelligence, people will only want sufficient insights to make a decision, and then want to go about doing their job. Augmented analytics will help surface those with micro-insights and “stories”, for the information consumer.
Hyperight: One crucial aspect of closing the gap between analytics experts and business users in the analytics supply chain is transforming data from “analytics-ready” to “business ready” data that has timely business logic and context. How can this be achieved?
Dan Sommer: Increasingly, we’re moving to a world where up-to-data data and real-time data is possible. The infrastructure is there, through advancements of IoT, and things like 5G, enabling us to be instantly-on. The tech is there through things like streaming data and real-time change data capture. But to become “business-ready”, we need that data to feel the pulse of the business, which can trigger actions at the right time. This can be done by setting up conditional alerts. Furthermore, as mentioned earlier, improved business logic will enable data producers and consumers to collaborate much before a dashboard is built.
Hyperight: How do you see the self-service analytics market changing in a post-COVID-19 world?
Dan Sommer: Our last discussion focused on the effects of COVID on the data and analytics market. You can find that discussion here. When we all work from home, self-service no longer cuts it. We also need self-sufficiency. Working from home, we’ve seen a strong need for a.) attractive user interfaces. b). But we’ve also seen that users often don’t want to self-serve, but increasingly expect insights to come to them. Expect more micro-insights and stories for the augmented consumer so that they can go about their business. c.) And also want access to the data, not just an application at the end of the chain that has been built for you.
Hyperight: Could you share what you are working on at the moment, and what does Qlik’s roadmap entail in the near future?
Dan Sommer: Right now, I’m deeply engulfed in our trends report for 2021. Our Data Revolution Tour keynote will cover these. Watch out also for a more extensive e-book and webinar on this topic! Usually, you can glean much of what Qlik is doing through this, as it’s our “view of the world”.
Add comment