July 23, 2021
Cloud TeamBusinesses are being forced to depend on human behaviour amidst the Covid-19 pandemic. In this blog, we will articulate how Machine Learning approaches can help with Demand Forecasting and sales predictions.
Demand is a consumer’s desire to purchase goods and services at given prices. Businesses put considerable effort and money to determine the amount of demand for their products and services. For instance, they create demand schedules to optimize the manufacturing of products and manage supply schedules. Economically, the demand schedule has a theoretical maximum, but the demand depends on several factors such as trends and multiple intrinsic & extrinsic factors.
Demand Forecasting is a process in which historical sales data is used to estimate customer demand. Critical business assumptions, such as turnover, profit margins, cash flow, capital expenditure, risk assessment and mitigation plans, capacity planning, etc., are determined by demand. It is a best-unbiased guess for the future sales/demand Or estimated future trends based on available information. In other words, Demand Forecasting is an art as well as the science of prediction.
It is based on past behavioral patterns and the present continuing trends, thus estimating the demand scientifically and objectively. If we address demand forecasting by determining it accurately, we get several benefits, driving better business decisions with insights into market demand, segment growth, and competitive dynamics.
The benefits of Demand Forecasting include:
Matches Customer Demand with Plant Production- Improves Inventory / Stock Management
- Improves Marketing ROI
- Facilitates Financial Planning, Pricing Policy, and Expansion Plans
- Improves Order Fulfilment
- Boosts Market Share
- Maximizes Gross Profit
The Forecast needs to be accurate as it poses an impact over several factors:
A business needs to get to the forecasting cycle basics to determine the forecast, which consists of determining forecast purpose, collecting historical data, selecting appropriate forecast models, computing forecast and checking the forecast accuracy. This forecast cycle is repeated multiple times.
A typical forecasting engine and the inputs and outputs can be represented as below:
While working with the forecasting cycle and engine, organizations need to follow the best practices of demand forecasting that critically impact the forecasting performance. These practices revolve around up-to-date sales and other data required. Read on.
- While making smart forecasts, data closest to real-time shall be used to avoid the demand calculation with any missing data points. This enables continual forecasting on a weekly or monthly basis with the latest available data/information.
- A repeatable monthly process must be created.
- Forecast accuracy must be consistently measured and adjusted accordingly.
- Organizations should focus on using the forecasting methods suitable for a particular product group (segment). At times, the drive for consistency will enable a business to use the same forecasting method for every item – this is a recipe for disaster.
Our team at Motherson Technology Services experienced a few challenges while working on demand forecasting, as explained below:
- At times, we are unaware of the exact factors- internal or external, impacting the demand. E.g., Harvesting calendar affects the demand for products.
- Interrelationship amounts to various factors that impact demand, and segregation of these factors as well as choosing primary factors are only a part of the analysis.
- Data quality is a significant issue. It is either incomplete or unavailable for the required duration.
- Several external factors which impact demand are not readily available.
- Granular data is scarce. Therefore, we need to aggregate or segregate the data depending on the results.
The team worked with a customer where we successfully applied the demand forecasting approach to gain results. The client wanted to save production costs by optimizing their sales forecasting and ensure a reformed production plan.
The challenge was to forecast the accurate sales of selected product models for the upcoming months based on several time-series datasets. Subsequently, they wanted to adjust the production plan of the products chosen. They previously relied on manual forecasting techniques with sub-optimal results.
We identified the following key points addressable with an ML solution as per our discussion with the line of business owners.
- To forecast the sales of the selected products for the upcoming months based on multiple time-based various factors, including macro-economic indicators
- Multiple data sources to be consolidated to get a holistic view. Few challenges with the data were:
- Data in multiple formats and different granularity levels
- A lot of missing data points and a need for numerous publicly available data sourcing
- Scarping was done as no defined interfaces were to download publicly available data
Based on our discussions with the customer, we examined various data sources and data sets that can help address the business problem and proposed an ML-based demand forecasting solution.
We received three years of data for selected products to model it for forecasting Sales Data, Enquiry Data, Seasonal holidays and others. The customer also provided secondary data, including Social Listening, Brand Track, Website Hits, Discounts & promotions for modelling demand.
As a part of the engagement, we carried out the following activities:
- We had scrapped economic indicators, competition, and holiday data from public portals. These economic indicators include Bank Credit Growth, Bond Yields, Capacity Utilization, RBI Repo Rate, Fuel Rates, Stock Price, CPI and GDP.
- We analyzed the data using multiple techniques, such as exploratory data analysis, feature selection, dimensionality reduction (PCA), regression analysis and PCA for dimension reduction.
- We created ML models for demand forecasting –
- The team applied several methods like various smoothing techniques, ARIMA, Deep AR models to map the demand.
- We created LSTM models to forecast sales of the selected products using a custom LSTM algorithm on SageMaker as it is less prone to variance and seasonality. However, due to limited data points availability, it led to subdued accuracy, leading to adopting a better model.
- Our team created a model using fbprophet, developed and used by Facebook for forecasting using Amazon SageMaker. We went ahead with this approach as fbprophet was giving better accuracy and results.
- We used spot instances as spot training can optimize the cost of training models by 90% over on-demand instances.
- We aggregated Invoice & Enquiry data monthly to form time series of sales and enquiry, respectively and did weekly aggregation to generate more data points.
- As there were 121 product variants in the data, many of the variant counts of data points were insufficient for forecasting. Therefore, we employed a criterion to choose the best variants for which the forecasting could be done.
- We took weekly data aggregation instead of monthly to increase data points. If taken monthly, we don’t get sufficient data points even for the selected product variants.
- We chose the variants for which the data was available for the previous year, i.e. ensuring the most recent data points.
- The Motherson Technology Services team worked on decomposing time series into constituents to observe components like seasonality and trends.
- We checked for various properties of the series, e.g. stationarity, using statistical techniques and data partitioning.
- We modelled the data for existing patterns using various statistical / Machine Learning models, e.g. DeepAR, ARIMA, smoothing techniques etc.
- We compared the results for various models and differentiated them by injecting other available data, such as competition data and other economic indicators.
- We chose the best accuracy model, forecasted with the model and compared it with a manual forecast.
- There is a need to improve models by mapping daily data for longer durations – for about 8 or 9 years to cover all possible business cycles, thus further enhancing forecast results. This is because the current business cycle is unique with respect to the data used.
- More parameters must be included, such as natural calamity situations/floods, to refine the forecast.
- The harvesting calendar for states needs to be comprehended.
- A weightage analysis for each factor influencing the demand should be included. Moreover, the focus must be to have more relevant parameters and apply combinations accordingly for secondary data parameters in a model, hence identifying the best-fit parameters.
- Forecasts shall be computed at the state level instead of an aggregated demand at the country level, increasing forecast accuracy.
In this blog post, we talked about how we worked on the demand forecasting technology and understood what added value it can provide to modern businesses.
After analyzing different ML models, we selected the one with which we carried out the sales forecasting with up to 93% accuracy. This helped us optimize the workforce at sales channels as per the demand forecast, allowing the production team to adjust the production plan based on the forecast.
After running various models and trying different algorithms, we decided to select fbprophet as the model, reducing the model training and deployment costs by using spot instances.
If you want to learn more about ML solutions, request a demo or contact us here.
About the Author:
Dr. Bishan Chauhan – AI / ML Practice Head
Abhinav Kushwah – Data Scientist, AI / ML Practice
Lakshay Malhotra – Data Scientist, AI / ML Practice
