HOW TO SCALE DATA SCIENCE PILOTS TO INDUSTRIALISE YOUR DATA: PHILIPP M DIESINGER

Through agile collaboration, small teams can rapidly develop promising Data Science pilots. However, scaling such minimal viable products (MVPs) to enterprise level is a complex process with many challenges along the way. But, with the right approach and collaboration model, these challenges can be overcome – resulting in industrialised solutions that lead to significant business value.

It has never been easier for organisations to leverage their data by developing tailored software solutions. Backend, frontend and data analytics frameworks are readily available, integrated data platforms have advanced and matured significantly and cloud environments provide readily-available capabilities for analytics, software development and
deployment. Developing tailored solutions to meet organisation-specific challenges can lead to competitive advantages and internal efficiencies.

While typically developed in small teams, Data Science solutions can be scaled up for large amounts of users. In contrast to other activities that provide business value, scaling Data Science solutions relies mostly on technology and not on manual labour, which represents a significant advantage. The ability to develop and scale Data Science solutions for relevant stakeholders across an organisation has become a critical capability for every medium to large-scale enterprise.

The process can be broken down into two major steps:
Step 1: The development and field-testing of a Data Science pilot or MVP.
Step 2: The industrialisation and scaling of successful MVPs to relevant users across an organisation.

Either of these steps can be achieved with the help of third parties, however, the initial creation of critical IP (Step 1), often takes place in-house.

While the industrialisation of a Data Science pilot depends on the situation (based on factors such as relevant data, user-types, available technology, business goals, etc), the process itself shows recurring patterns and phases across use-cases, organisations, and even industries – as illustrated in the chart.

Before industrialisation: start of industrialisation

After a Data Science pilot has been developed and tested, the organisation can then decide whether to scale-up, industrialise and roll out the solution to relevant stakeholders. This decision typically depends on a number of relevant factor such as the expected business impact in light of the pilot’s field-test results, budgetary and capacity constraints, the degree of complexity added to IT systems and the required level of change management. Ideally, these topics have already been considered during the pilot development. In any case, a tested MVP allows a more accurate assessment which can then lead to an informed decision for – or against – industrialisation.

PHASE ONE – ALIGNMENT

Once the MVP has been approved, the industrialisation process can start. Typically, first steps include expanding the team of relevant collaborators by identifying stakeholders who need to be involved, plus an initial alignment with technology owners as well as the setup of a healthy collaboration model.

Early in the process, it’s important to discuss and agree on a shared vision. This should include agreeing on transparent success metrics and measurement frameworks for the desired business value the solution is expected to deliver to the organisation.

Industrialising an MVP is a highly collaborative effort which requires a significant degree of alignment between stakeholders with varying responsibilities across different parts of the organisation. Stakeholders typically include: the business departments benefiting directly from the solution (by consuming the datadriven insights), the Data Science teams involved in the development of the initial pilot, the IT departments required for integrating the solution into the techstack, the departments responsible for running and maintaining the solution. Other stakeholders may include data governance, master data management, legal, compliance, cyber-security experts, GDPR officers, ethics experts and important external thirdparties.

It can be challenging to align such a varied group of collaborators. A well-defined industrialisation scope, as well as clearly defined responsibilities, milestones and timelines can help significantly.

PHASE TWO – SETUP

Once the industrialisation process has been aligned, a more formal project setup can then be established. Nominating the right stakeholders into project leadership roles with short communication channels can be very beneficial (and it’s important to not leave out crucial stakeholders in the process). Often, challenges arise from miscommunication or missing alignment between different departments, as well as a lack of transparency or poor expectation management. Strong and early alignment can prevent such potential problems from becoming serious issues later in the process. Choosing and setting up the right collaboration model and collaboration framework can be equally important. The goal of establishing the right setup for an industrialisation process is to meet all the requirements needed for a successful execution phase. This typically includes finalising the scope, agreeing on clear deliverables and timelines, establishing collaboration roadmaps, creating awareness of potential risks, agreeing on platform and technology topics as well as establishing transparency on user types and user profiles.

Once the requirements become clear and transparent, it can be a good time to perform capacity planning of the involved stakeholders. It is important to have realistic timeframes and a roadmap that retains some degree of flexibility for unforeseen events and challenges.

An industrialised solution can only benefit the organisation if it is accepted and used by the target group of end users. Therefore, discussing the degree of change management as well as concrete steps to make the solution convenient to use can be of significant value.

PHASE THREE – EXECUTION

Once alignment has been reached and the industrialisation process set up, the main phase of the industrialisation process can start: the execution phase. At this point, ideally all stakeholders have a clear understanding and transparency on deliverables and timelines; plus they know who their partners are and how to align and communicate with them. As the industrialisation process is a highly collaborative effort that requires many different skill sets, it is important to keep timelines and roadmaps realistic and to communicate issues and delays early and transparently.

Industrialising data requires technical expertise across multiple areas. Adding a new component to a technology stack or platform increases its complexity often exponentially because connections to existing components need to be established. For instance, where a new node is added to an existing complex IT network. Therefore, it can be beneficial to consider re-using existing capabilities as much as possible. Especially, new APIs or ETL processes can increase complexity and cost substantially. Utilising datacentric setups can be key for making sure a solution experiences a long lifespan.

The industrialisation of a new Data Science solution can also present a great opportunity to deal with already problematic legacy systems, as a measure of hygiene for the entire tech-stack. During industrialisation, technical topics are often the dominating challenge, but it is important to keep the user base in mind as well. For instance, an end usercentric development model for components like the UI can significantly lower the bar of the required change management later. Including and empowering users, providing them with a degree of agency and a feeling of co-creation can be very beneficial.

AFTER INDUSTRIALISATION: RUN-MODE

Once the solution has been industrialised it can be released into run-mode. This entirely new phase of the life cycle comes with its own challenges, such as monitoring and maintaining new technology as well as managing the needs of end users.