Four steps on how to build a machine learning solution

Nowadays there is a lot of talk about artificial intelligence (AI) and data. Many organizations have acquired their own data scientist and competition for smart solutions is fierce. At the same time, however, it has become clear that building solutions that work with data, AI or machine learning models is laborious and leads to point solutions.

In this blog, I will go through four steps on how to build a machine learning solution that successfully utilizes artificial intelligence methods, as well as what aspects need to be considered when designing this type of project. To make things clear, examples will be offered for each step.

1. Find a problem that really should be solved with machine learning

The most important step in an AI project is to start with a well-defined and crystal clear business problem. Ask yourself whether the problem really requires machine learning and can you tolerate the fact that algorithms may not provide the solution that was expected. A machine learning solution is not worth the investment just because competitors say they do it - often with a smart data solution, many things can be built far better than by forcing the use of algorithms.

An example of a good problem to be solved could be; the recommendation provides improved sales/profitability by X percentage points, whereby the investment pays itself back in N months.

2. Ensure you have the right data and capabilities

The second step is ensuring you have sufficient data capabilities. For example, having an existing data infrastructure or constructing it for the project and future, as well as having historical and sufficiently diverse data and/or availability to third-party data for the machine learning models. 

Researching, collecting, and analyzing data usually takes a lot of time. A lot. More than what you would expect. However, all the information available to the algorithms resides in the data, which is why this step is in a critical role.

For example, you need to handle sparse history data from which you need to consider changes in, for example, the product hierarchy, source system, and customer behavior over the history of the data. For a quick start, you can export your own Google Analytics data and use that to recommend your items. However, a better way is to build your own pipeline for the data collector and centrally store different data in one place.

3. Model > Test > Validate > Repeat

The third step is modeling. After a well-done data phase, the development of a machine learning model is quite straightforward. Today, many tools, cloud environments, and programming languages have their own frameworks for implementing the machine learning algorithm. The key is to understand which algorithms are best suited for the problem. However, more iterations are typically required between data editing and different algorithm options before the best possible result is obtained.

 Once the model is ready and has been tested against the test set and the solution is technically functional, it should still be validated in the production environment, i.e. in the environment where the solution will be utilized. In this context, it is important to consider for example the market situation, business cycles, and the fact that new outputs may influence the behavior of the users. Therefore, testing should be designed to give the most realistic picture as possible of the solutions’ functionality. It is not always simple, but it is the only way to ensure that the solution has a genuine business benefit.

For example, you first train the most appropriate algorithm based on intuition, after which you test the predictions against the test data. You then modify the source material, experiment with different hyperparameters as well as other approaches appropriate to the problem, and validate the most appropriate combination for user validation based on the results. Once it’s done, you’ll do the same thing again and again a little better than before. You continue this until the model no longer improves much.

4. Make sure that the solution is integrated into business processes

The fourth and final step consists of integrating the solution into business processes and ensuring continuity. This theme has recently been featured in the form of MLOps discussions. To ensure the implemented machine learning solution becomes a real part of the business, the solution must be automated so that the model remains up to date, it has the necessary alerts to ensure the quality of operation, and informs on the accuracy of the level of prediction at any given time.

However, technical capabilities alone are not sufficient to provide the best possible benefit from the implemented algorithm. Managing people, operating methods, and expectations are also key to utilizing the output of the solution. A tool or solution is completely useless if it is utilized with insufficient capacity. The organization needs to harness the solution and ensure that it is also utilized and further developed by the users.

For example, you incorporate the recommendation algorithm into business processes so that the end customer truly receives the benefits of the recommendations. At the same time, you ensure that the staff understands what kind of entity it is and are able to utilize and further develop the algorithm for future business needs. Not forgetting, of course, the closed-loop, which ensures that the impact of recommendations and changes in consumer behavior are taken into account in retraining the algorithm. This in turn allows for continuous learning.

A short recap on how to build a machine learning solution

A data-driven business solution consists of the following components:

1) Business case design

2) Data and its processing

3) Model, test, and validation

4) Integration into business processes and ensuring continuity

Each step is in itself an important part of the process. A small part of the machine learning solution can be tested with a proof of concept or a pilot, but these only give the partial truth about the full potential of the solution.

-

Want to take your company’s AI capabilities to the next level? We at Recordly help companies create profitable data solutions. Contact us and we will have you covered in no time. 

In the meantime, if you’re keen on learning more on how to utilize data and AI within your team, download Futurice’s Data & AI Handbook. The handbook offers 10 practical steps to guide your team through the process of Artificial Intelligence.