Beyond the Hype – Part 1: Machine Learning Powering Real-World Solutions

The series consists of three posts, with the first laying the groundwork: what are we even talking about here, and why should I care? The second one builds upon this by touching on the technical requirements for applying advanced analytics into your daily operations, with the third one focusing on organization-wide AI adoption for strategic growth; a non-technical approach. 

We specialize in all three topics covered; should you be interested in either mapping your data/AI capabilities or even implementing your own solution, do not hesitate to contact us.

The artificial intelligence (AI) landscape today is largely dominated by discussions of generative AI (GenAI). Headlines about chatbots that can summarize reports, write code, or generate images have captivated the public’s imagination. As a result, there is a common misconception that AI is primarily about content creation, prompting, and summarising and that the dawn of AI as we know it was in Nov. 2022. Furthermore, it seems that during AI/data-related discussions, if I ask people ‘Why generative AI?’, they usually look back at me intrigued. They do not fully get the question. “Obviously, GenAI, that’s the one-solution-fits-all silver bullet!” Others mention a strategic objective identified by the board or a C-level executive. 

More often than not, however, there are better answers than GenAI to address their business priorities. Figure 1 further illustrates the landscape and, thus, the possibilities of advanced analytics and machine learning (ML). However complex the topics may get, it is crucial to remember the why; we’re essentially automating the process of creating conclusions out of data - transforming data into information for us to derive knowledge to make more intelligent decisions (Figure 2).

AI & ITS SUBCOMPONENTS

Figure 1. AI and its subcomponents [1].

While there is a lot to unpack and a lot of complex terminology, the main point here is that GenAI is but a subset of deep learning (DL), represented by Transformers and Large Language Models (LLM’s). The categories are as follows:

1. Artificial Intelligence (AI), where the basic idea is to enable a machine to imitate human behavior and cognition. This also includes achieving complex processes such as problem-solving, learning from experience, adapting to new situations, recognizing patterns, understanding language, and even making decisions.
   
   
2. Machine Learning (ML), which is essentially a fancy word for statistical modeling, often extends on and automates the kinds of modeling that traditional statistics have always done. While statistical models tend to emphasize inference and understanding relationships within data, ML leans more towards prediction accuracy and scalability with large data sets.
   
   
3. Deep Learning (DL), a subset of AI that processes data in a way that is inspired by the human brain and learns from vast amounts of data. The key difference between ML and DL is the way the model learns from data. ML utilizes statistical properties in data and, as such, does not generally work well on complex and large datasets. DL, on the other hand, utilizes calculus and a technique called backpropagation to minimize an error function through gradient descent, which allows the model to infer more complex relationships from larger amounts of data.
   1. Unsupervised learning, a subset of ML, infers relationships between similar clusters of data without explicit labels. For example, given the sizes and colors of pumpkins and pears, the system would learn that smaller and greener inputs are more similar to one another, as opposed to the characteristics of a pumpkin. 
   2. Another subset of supervised learning involves training a model on a labeled dataset, where each example in the dataset is paired with an output label. This approach involves learning the relationship between input features and known target outputs, allowing the model to accurately predict outputs for unseen data.
   3. The third, maybe slightly more unknown and advanced subset, is semi-supervised learning. Semi-supervised learning combines a small amount of labeled data with a larger pool of unlabeled data. In this setup, the initial supervised model is trained on the labeled data. Once it has learned from this limited set, it can apply its own labels to the unseen (unlabeled) data, effectively becoming a "teacher" to itself. This effectively solves the problem where collecting labeled data is expensive, difficult, or both, but collecting similar data without labels is relatively straightforward. 
      
      
4. Reinforcement Learning (RL), where the main idea is that an agent learns by interacting with an environment, receiving feedback in the form of rewards or penalties based on its actions. The goal is for the agent to learn a policy that maximizes cumulative rewards over time, refining its strategy through trial and error. This iterative process helps the agent develop a deep understanding of the environment, often leading to highly optimized behavior.

Figure 2. Data as the basis of operations; DIKW-pyramid [5].

TIME SERIES FORECASTING

Although not present in Figure 1, there’s another ML-related subject called time series forecasting. As the name suggests, time series forecasting is the statistical way of predicting the future. It answers the question of ‘Based on past experience, what will happen n-time into the future?’. Time series analysis, on the other hand, answers the question of what happened and why - a form of descriptive analytics. For example, we might want to project the amount of ice cream sales based on weather and analyze the relationship between ice cream sales and weather. In other words, on average, for each unit of temperature rise, this many more ice creams were sold, or based on tomorrow's weather forecast, this much ice cream is expected to be sold. Figure 3 depicts an example of time series forecasting on a yearly scope by Susan Li [2]. However, time series forecasting can be done down to the minute, or even second, should your data be granular enough. For example, in high-frequency trading (HFT), the buy/sell -orders for a commodity or a stock goes down to the millisecond level. 

Figure 3. Time series forecasting example [2].

Predictive and prescriptive analytics have been at the core of data-driven decision-making for years, transforming operations and helping businesses preemptively address challenges. Although the term ‘AI’ was first coined in 1956 in Dartmouth’s Summer Research Project [3], it was not until the early 00’s and onwards that advanced analytics began to see wider industry adoption, mostly thanks to advancements in computing power and the general availability of data. 

Predictive analytics uses historical data to forecast future outcomes, enabling companies to stay ahead of potential challenges. Take, for instance, the finance industry, where predictive models are fundamental for assessing risk, detecting fraud, and forecasting market trends. Banks and insurance companies rely on these models to anticipate risks and allocate resources effectively, saving millions and protecting customers in the process. In manufacturing, predictive analytics have revolutionized demand forecasting and maintenance. By predicting equipment failures, manufacturers can schedule repairs before breakdowns occur, reducing downtime and extending the life of costly assets. Retailers, on the other hand, use predictive analytics to understand customer buying patterns, tailoring stock levels to meet demand and preventing overstock or stockouts. 

While predictive analytics tells us what’s likely to happen, prescriptive analytics takes it a step further by recommending the best course of action based on these predictions. Imagine a retail supply chain facing volatile demand and fluctuating prices. Prescriptive models can optimize stock levels, distribution schedules, and even supplier choices, ensuring cost efficiency and customer satisfaction. Essentially, the question answered here is ‘Given situation A, what should we do?’. In finance, prescriptive analytics can support investment strategies by providing recommendations based on market conditions, risk appetite, and historical performance data. Similarly, manufacturing companies use prescriptive analytics to streamline operations and improve resource allocation. 

SUPERVISED VS. UNSUPERVISED

Not all applications are based on predicting the future, however intriguing that might sound. Imagine, for example, an ad campaign. Despite knowing the most optimal time to show advertisements, through time series analysis, for potential customers, if the customer cohort you’re targeting is off, you’ll end up wasting both time and money; possibly even irritating your potential customers. Here’s where classification comes in: by grouping users, either with supervised or unsupervised methods, into similar cohorts, you can not only optimize the time of day when to show an advert, you’ll also be able to show it to the most optimal crowd. This saves resources, improves customer satisfaction, and drives revenue. 

Figure 4. Supervised vs. unsupervised learning [4].

The AI landscape is complex and evolving at a rapid pace. It is not always easy to identify which approach is the best fit for a given business problem. In some cases, AI is not even the answer, however wild that may sound. The key to success is for domain experts and ML experts to sit down for a discussion in order to align business-related requirements with technical ones - to help the experts understand the problem at hand and the domain experts how advanced analytics might help solve that problem. Here at Recordly, for example, we’ve got specialists in statistics, ML/DL, GenAI, and data engineering, all eager to help you bring data-driven insights to reality. Here are, however some key points to think about when planning your next project:

• What is the business problem, and what kind of output is needed? Predictive models are usually a good fit for a numeric or classification prediction, while generative models excel in content generation (text, image, video, audio);
  
  
• What data is available to address the problem? Structured and labeled data can easily be leveraged using predictive models. A large corpus of text is often a better fit for generative models;
  
  
• Are you more looking for accuracy or creativity? Predictive models prioritize accuracy, while generative AI focuses on creating novel outputs;
  
  
• Do you have strict explainability requirements? Predictive models are often more interpretable, while generative models can produce more creative results that are difficult to explain; further, statistical models are generally more interpretable than deep learning models;
  
  
• Can you benefit from a hybrid approach? GenAI is not the answer to every problem, nor is predictive AI. Sometimes, combining the two is the best approach; maybe you’d like to classify your customers based on behavior and use GenAI to generate custom marketing? Perhaps your business would see value in automatically creating supply orders for next week based on time series forecasting? The possibilities seem endless.

Theory becomes truly compelling when supported by real-world examples. Hence, we’ll end this blog by exploring detailed case studies from industries that have successfully implemented advanced analytics and AI to drive transformation and create business value. 

The first case, which has effectively created a real opportunity to use ML, is one with our customer Lumo Analytics. We’ve helped them utilize domain-specific data by implementing an Internet-of-Things (IoT) data platform which enables Lumo to gather and store sensor data from the field and turn it into actionable insights. While this is not ML per se, a robust data platform and strategy are crucial for advanced analytics and will, already as such, drive real value. More on such requirements will be in the following blog post, so stay tuned!

Further, once a robust data infrastructure is up and running, one might utilize time series forecasting to kick their analytics up a notch. Take Verkkokauppa.com for example, who not only entrusted Recordly with building their entire Machine Learning Operations (MLOps, more here) platform, but also improving their day-to-day operations by predicting stock levels of various items; with the right amount of the right products on the shelves, not only are the costs of logistics reduced, but both customer satisfaction and revenue go up. Pretty cool stuff.

To stay true to the times we’re living in, this would not be much of an AI blog without mentioning GenAI. Hence, introducing Alma Media, who together with Recordly and Futurice, found a use case for GenAI. Utilizing LLM’s, Recordly and Futurice enhanced Alma Media’s search engine by continuously processing and updating new articles as they are added to the system, ensuring robust and efficient search capabilities.

While these cases illustrate that data-driven insights are more than just “nice-to-haves”, there are countless other use cases for advanced analytics; they are strategic assets that help companies stay competitive, adaptable, and efficient. By prioritizing the "why" of AI adoption—solving meaningful problems and uncovering data-driven opportunities—business leaders can transcend the hype and leverage ML as a powerful, transformative tool.

In the next blog, we’ll discuss the actual technical requirements needed to bring your AI vision to life! However, if I’ve managed to spark your curiosity, and you’d like to further discuss or explore use cases specific to your business, please do not hesitate to contact us - we’re here to help you take your business to the next level.

we at recordly help your organization harness data & AI/ML in a way that actually makes sense for YOUR business.

References

[1] Hanassab, S., Abbara, A., Yeung, A., Voliotis, M., Tsaneva-Atanasova, K., Kelsey, T., Trew, G., Nelson, S., Heinis, T., & Dhillo, W. (2024). The prospect of artificial intelligence to personalize assisted reproductive technology. npj Digital Medicine, 7, Article 10.1038/s41746-024-01006-x. https://www.researchgate.net/figure/The-artificial-intelligence-landscape-A-Venn-diagram-providing-a-holistic-view-of-the_fig2_378676909

[2] Li, S. (2018). An End-to-End Project on Time Series Analysis and Forecasting with Python. Towards Data Science. https://towardsdatascience.com/an-end-to-end-project-on-time-series-analysis-and-forecasting-with-python-4835e6bf050b

[3] Dartmouth College. (n.d.). Artificial intelligence (AI) coined at Dartmouth. Dartmouth College.  https://home.dartmouth.edu/about/artificial-intelligence-ai-coined-dartmouth

[4] SuperAnnotate. (2023). What is image classification? SuperAnnotate. https://www.superannotate.com/blog/image-classification-basics

[5] Wikipedia. (2024). DIKW pyramid. Wikipedia. https://en.wikipedia.org/wiki/DIKW_pyramid