Despite the growing need for AI to bring a newfound agility to a post-pandemic world, businesses still struggle to pivot their operations around data science, machine learning (ML), and AI technologies precisely because it’s not simply a matter of technology; processes and people are also critically important. Let’s focus on the role people play in achieving MLOps (short for machine learning operations), a process that brings the required agility and allows for massive scaling of AI initiatives across the enterprise. MLOps is the standardization and streamlining of ML lifecycle management—it pulls heavily from the concept of DevOps, which streamlines the practice of software changes and updates. In other words, it helps organizations and business leaders generate long-term value and reduce risk associated with data science, ML, and AI initiatives.
A simple representation of the machine learning model life cycle, which often underplays the need for MLOps (image courtesy O’Reilly’s Introducing MLOps).
First of all, successful data teams and data projects involve experts in IT, business, and data science from the start. Pulling in expertise at the last minute when most of the work is already done is extremely costly and is a sign of larger organizational issues around AI projects. At Dataiku, we believe that the use of data and AI should be everyday behavior for everyone, powering collective success. In order to truly achieve everyday AI (and everything involved in it, such as MLOps, for example) everyone needs to be on board. Succeeding in AI initiatives requires fostering a culture of data creativity at the individual level and then finding a way to harness that individual data creativity to power collective company progress and purpose.
The AI project life cycle must involve different types of profiles with a wide range of skills in order to be successful, and each of those people has a role to play in MLOps. But the involvement of various stakeholders isn’t about passing the project from team to team at each step—collaboration between people is critical. For example, subject matter experts usually come to the table—or at least, they should come to the table—with clearly defined goals, business questions, and key performance indicators (KPIs) that they want to achieve or address.
A representation of who is responsible at different levels of the organization for different parts of the Responsible AI process (image courtesy O’Reilly’s Introducing MLOps).
In some cases, they might be extremely well defined (e.g., “In order to hit our numbers for the quarter, we need to reduce customer churn by 10%,” or “We’re losing n dollars per quarter due to unscheduled maintenance, how can we better predict downtime?”). In other cases, less so. (E.g., “Our service staff needs to better understand our customers to upsell them,” or, “How can we get people to buy more widgets?”)
In organizations with healthy processes, starting the ML model lifecycle with a more defined business question isn’t necessarily always an imperative, or even an ideal scenario. Working with a less defined business goal can be a good opportunity for subject matter experts to work directly with data scientists up front to better frame the problem and brainstorm possible solutions before even beginning any data exploration or model experimentation.
The realistic picture of a machine learning model life cycle inside an average organization today, which involves many different people with completely different skill sets and who are often using entirely different tools (image courtesy O’Reilly’s Introducing MLOps).
Subject matter experts have a role to play not only at the beginning of the AI project life cycle, but the end stage of post production as well. Oftentimes, to understand if a ML model is performing well or as expected, data scientists need subject matter experts to close the feedback loop—traditional metrics (accuracy, precision, recall, etc.) are not enough. For example, data scientists could build a simple churn prediction model that has very high accuracy in a production environment; however, marketing does not manage to prevent anyone from churning.
Monitoring and feedback loop highlighted in the larger context of the ML project life cycle (image courtesy O’Reilly’s Introducing MLOps).
From a business perspective, that means the model didn’t work, and that’s important information that needs to make its way back to those building the ML model so that they can find another possible solution on a strong foundation with MLOps, it might be worth looking at the steps AI projects must take at your organization and who needs to be involved. This can be a good starting point to making sure the right stakeholders not only have a seat at the table, but that they can effectively work with each other to develop, monitor, and govern models that will not put the business at risk. To learn more about MLOps and how to leverage it to bring value from a business perspective, check out this ebook.
Top image via Dataiku.
