Future-Ready MLOps: Strategies for Success in Coming Years

In 2024, MLOps (Machine Learning Operations) emerged as a key focus for leaders across many organizations. It gained more attention than ever before. The landscape of Machine Learning (ML), and Artificial Intelligence (AI) more broadly, has become fundamental for various organizations. This includes those in retail and healthcare. AI is also integral to people’s everyday lives [1].

With businesses becoming more deeply integrated with ML solutions, the demand for effective MLOps practices has grown significantly. As a result, the MLOps tools landscape has undergone a major transformation to keep up with these rising business needs.

But… What is MLOps?

MLOps practices aim to support data-driven solutions. These solutions are built in a collaborative, efficient, and scalable manner. When possible, MLOps also provides a self-service experience. This helps reduce inefficiencies and eliminate unnecessary processes.

Facilitate Cross-collaboration

Provide the necessary support to enable seamless collaboration among team members. This includes Data Scientists, ML Engineers, Data Engineers, and MLOps Engineers. They work together on ML solutions.

Speed up Development and Time-to-market

Given its exploratory nature, quite often ML solutions are not properly designed to provide a fast deployment.

Build Scalable and Robust Solutions

Design ML solutions with scalability in mind to prevent scenarios where solutions fail to fit within a larger framework, avoiding the need for extensive rework.

Promote Reproducibility

• Minimize redundant efforts by designing solutions that prevent the need to rebuild components repeatedly and inefficiently.
• Have proper governance to allow ML solutions to be re-run and produce consistent outcomes

Optimize Resource and Costs

Design solutions that facilitate efficient resource use (e.g., through distributed computing) and enable effective cost control

As the year draws to a close, it’s time to reflect: how far have we come in the MLOps space?

To better address this question, The Institute for Ethical AI & Machine Learning has launched a survey titled “The State of Production ML in 2024.” In Figure 1, we can see the responses to the top three challenges in productionizing ML solutions. A quick look reveals that there is no clear consensus, as the challenges are almost evenly distributed across approximately seven different topics. This indicates that organizations are still progressing toward achieving maturity in their ML practices.

The MLOps technology landscape emphasizes enhancing the deployment of ML solutions and monitoring their performance. It also facilitates a smooth experience for experimenting with and comparing various model versions. Without these advancements, organizations risk relying on manual, ad-hoc methods that ultimately lead to inefficiency. According to the survey, illustrated in Figure 2, MLFlow [3] is the most popular tool for model registry and monitoring (tracking), with approximately 42% adoption. However, over 21% of respondents report minimal to no model monitoring, highlighting a diverse range of technology adoption. This variety of tools indicates that we are still in an experimental phase regarding MLOps tool adoption. Additionally, Figure 2 reveals varied feedback regarding the tools used for building ML solutions. Databricks and Amazon SageMaker are the most dominant tools, yet there is also a noticeable presence of custom solutions. Furthermore, about 15% of respondents do not have a clearly defined tooling strategy.

¹The Institute for Ethical AI & Machine Learning represents a Europe-based research center that develops frameworks that support the responsible development, deployment and operation of machine learning systems. It is formed by a cross functional team of volunteers, including leaders in technology, ML, industry and academia.

Strategies for MLOps Success

It begins with people and data strategy, respectively access to data and its governance. While this is not specific to MLOps, it is a critical dependency – if not properly managed, MLOps efforts are unlikely to succeed. Adopting MLOps practices and selecting the right technologies to fit the organization’s needs can provide numerous benefits, but there are several critical considerations to keep in mind:

1. What is required to get started?
2. What potential risks might arise?
3. What is needed to maintain consistency over time?

Figure 3 highlights five fundamental components required to adopt an MLOps strategy within an organization. When it comes to choosing MLOps technology, the decision is typically tied to four key components: 1) Orchestration, 2) Model registry and experiment tracking, 3) Model deployment and serving, and 4) Model monitoring. These components collectively focus on the development of ML solutions, automating their execution, and adopting software development best practices – such as versioning not only code but also models and the data used throughout the experiment lifecycle. Ultimately, these components enable seamless integration with other technologies, facilitate the deployment and serving of model outputs, and support the monitoring of models in production, ensuring that their performance is effectively tracked.

In Figure 3, we do not see people being highlighted, however it is a fundamental part to consider when going through any digital transformation. Going back to the first question, what is required to get started? People and data strategy!

In an organization, workstreams are often product-led, especially when following a Data Mesh strategy. Within each product team, members bring diverse areas of expertise, and when focusing on ML solutions, various specialized roles emerge, as shown in Figure 4. It is crucial to address the specific needs of these roles by considering questions such as: How will the MLOps strategy support them? Do they require additional training? Is there a need to hire more experienced professionals to strengthen the team?

A common risk when focusing on technology is vendor lock-in. This can create challenges if the chosen technology fails to support all business areas or has limited capabilities. Decisions should strike a balance between achieving quick wins and maintaining a holistic vision for future needs. Quick wins can drive adoption and highlight the benefits of the MLOps strategy. At the same time, a long-term vision helps avoid choices that lead to technical debt. Technical debt could ultimately hinder the organization’s progress toward a mature MLOps workflow.

On the other side of the coin, a significant risk lies in poor decisions regarding the development of ML solutions. For example, notebooks are widely used in the ML space due to their ease of use for data exploration and visualization. This has driven their popularity. While there have been many efforts to make notebooks more aligned with software development practices, they remain limited when it comes to implementing MLOps best practices. Particularly within a CI/CD workflow that addresses all the fundamental components outlined in Figure 3. 

So, what might a high-level architecture for an ML solution look like? Figure 5 offers a conceptual design. In this architecture, components are tracked through repositories, with meta stores for models and data, serving layers, and monitoring in place. The key takeaway is that these components should not be overlooked. They are essential for the development of every ML solution and should be adapted according to the specific needs of the solution and the product it serves.

So far, this article has focused on the key strategic decisions required for the successful adoption of MLOps practices, but maintaining consistency over time is equally important. The MLOps space is dynamic, with rapid changes in workflows, technology, and business needs. We must recognize this reality and prepare to adopt well-founded changes that align with business objectives. Additionally, automating and monitoring as much as possible across all deployed solutions is vital. When this way of working becomes an integral part of your team, you are likely on the right path to achieving long-term success. To conclude, Figure 6 provides a holistic view of the key elements for a successful MLOps strategy.

About the Author

Filipa Peleja is a Lead Data Scientist with a rich background in machine learning, a solid foundation in business acumen, and a dedication to advancing diversity in technology. She began her academic journey in computer science, before pursuing a PhD specializing in machine learning. Driven by a passion for bridging technical knowledge with business strategy, Filipa furthered her education with an MBA, equipping her to address complex technical challenges with a strategic, industry-oriented perspective. Her professional experience spans global organizations, including the LEGO Group, Levi Strauss & Co. and Vodafone. Filipa also leads a specialized bootcamp for women+ transitioning into technology.

Don’t miss Filipa’s presentation at the NDSML Summit 2024. Discover the strategies that will make your MLOps future-ready and set you up for success in the coming years. For more insights on this topic, tune into her summit talk and gain the knowledge you need to stay ahead!