DataOps has already gotten the epithet “enterprise data management for the AI era”. It transforms the technology and culture of an enterprise to foster data trust, protection, cycle times, insights, and cost-effectiveness, enabling a more agile and seamless data analytics. However, it goes well beyond data management for the AI enterprise. As Manuel Brnjic, DataOps Team Lead at Walter Group explained, DataOps is not just about AI, ML or Data Science. It is the data-driving license for companies willing to make data their first-class citizen. The whole organisation sees benefit in DataOps adoption.
Manuel will be talking more about this topic in his Data 2030 Summit session on How to structure a modern DataOps team. In this interview, we talked to him about some pertinent points related to his talk, such as his most significant learnings with DataOps, its role in establishing successful data governance, the reasons why companies fail with DataOps and other trends that will mark 2021.
Hyperight: Hi Manuel, we are super thrilled to have you at the 5th online edition of the Data 2030 Summit. As a start, please tell us a bit about yourself and your role at Walter Group.
Manuel Brnjic: Thanks for inviting me to the Data2030 Summit. I’m already looking forward to talking to a huge audience of lots of IT and Data Professionals. I started almost 5 years ago as Solution Architect at Walter Group. While taking care of the technical architecture for various projects, I also started serving our Data Science Team as kind of a single-point-of-contact.
Soon I realised that it’s not just about putting some ML models or algorithms without proper governance into production. I started to search the web and attend conferences that cover AI and Data Science to figure out more about how others tackle the challenge. That’s how my DataOps journey actually started.
Hyperight: DataOps is the data management for the AI era. Do you agree with this statement? And why DataOps is the right methodology for every company striving to be AI-driven?
Manuel Brnjic: I would even go a little bit further: DataOps is the data-driving license for companies willing to make data their first-class citizen. It is not just about AI, ML or Data Science. The whole organisation benefits from the adoption of DataOps. Users nowadays want high quality and consistency in data over the entire application landscape. In the end, they need to be able to trust the reliability of the data they interact with.
Better collaboration across teams, technology, and environments is what everyone expects from DataOps. But there’s more hiding behind the new buzz word: you will be able to do rapid innovation and experiments delivering new insights to your users with increasing velocity.
Hyperight: Your Data 2030 Summit session is on How to structure a modern DataOps team. What are the most significant learnings you got away from building a DataOps team and implementing DataOps into an organisation?
Manuel Brnjic: Organisations are different when it comes down to structuring their teams. Usually, there are several groups wrangling with data to provide some kind of value or insight. Controlling, Data Science or the Business Analysts, to name just the most common ones. It’s very important to get in touch with all of them and really understand how they work and what tools they’re used to in their daily routines.
The biggest challenges are not technical ones. You quickly learn that there are multiple ways of interacting with data. Don’t try to get each team down to the same set of tools or practices. Making sure that data is consistent and correct across the whole application landscape is key. If people can rely on your data, they’ll start to trust it.
DataOps is the data-driving license for companies willing to make data their first-class citizen. It is not just about AI, ML or Data Science.
Today I can definitely say that it is not enough to simply create a DataOps “team”. It helps to start with such a team and begin onboarding colleagues ensuring they all know your vision and help to transform the organisation. Over time you realise that you need a whole bunch of different skill sets and roles. That’s by far the most interesting part of forming and evolving DataOps within an organisation. In February I will share more details at the Data 2030 summit about it.
Hyperight: How DataOps helps organisations with establishing successful data governance practice?
Manuel Brnjic: With a proper DataOps culture in place, you quickly start adopting a decentralised data governance structure. There’s not one department or one guy that has to take care of it. Usually, you have several product owners within an organisation. Why shouldn’t you also have several data product owners depending on the data domain they’re familiar with? Those people can, of course, have a different main role but additionally being the ones ensuring proper data governance. If you have a couple of them per domain, it’s a simple job, as they’re anyways interacting with that data on a daily basis.
Hyperight: DataOps is a great way for companies to operationalise their data and analytics pipelines and get valuable insights in a faster and more agile way. But in practice, many companies are failing in deploying DataOps. Why do you find that is?
Manuel Brnjic: By simply creating a DataOps team, you won’t achieve anything. Everyone interacting with data needs to adopt the data-driven mindset. It helps if you have at least one ambassador who is proactively and constantly interacting with all necessary departments of an organisation. Ideally, he has a profound understanding of what DataOps actually is and forms a vision with his core team.
Companies are failing because in their DataOps initiatives they’re setting goals like “implementing a data catalogue” and they think that all will be fine. If it were that simple, the new buzz word DataOps wouldn’t exist. You need a group of people that is really going to change the way of thinking about data in the overall organisation. Nowadays, tech landscapes are getting more complex. Just as we had to learn how to cope with COVID-19, it is important to start coping with the growing complexity. There’s no other way – you have to start thinking about your data proactively before it gets a mess.
Hyperight: DataOps is listed as one of the trends that are going to disrupt data and analytics in 2021. What are some other trends that we can expect?
Manuel Brnjic: Besides DataOps, I think that the democratisation of IT is going to be one of the biggest trends. Ten years back, it was simply impossible to call yourself a programmer if you had no Software Engineering Degree or an extensive amount of experience. Today we are not just talking about all those Data Scientists or Mathematicians who start implementing software on their own – at least parts of it. Basically, everyone can start programming without the need for extensive and costly training. A lot is open source, and there are huge communities and tutorials on how to get started with some easily learnable frameworks or no-code/low-code platforms.
In our fast-paced world, it’s impossible to wait for the “proper” implementation of every single thing. If everyone has to wait until his 100 LoC script gets implemented properly by “the IT guys” we would get nowhere. IT architects and leaders need to think about possibilities of how to interact and cope with those “wild animals” produced by all kinds of employees – not just the IT departments. Maybe a small set of rules those people have to comply with is a good start. In the end, it probably also results in having a couple of IT people proactively controlling the usage of popular low-code/no-code platforms within an organisation or even providing/implementing them! Think about providing a robust framework that is easily usable by everyone to change some behaviour or implement additional features in your application landscape, just like implementing Excel Macros or simply using the Functions.