What data is needed for forecasting software?

The data collected by companies can be exploited in different artificial intelligence applications such as sales forecasting, improving customer experience, reducing storage costs in retail, optimizing permanent or promotional sales, forecasting customer traffic in physical, web and mobile points of sale, etc. One of the methods used to exploit data to improve the achievement of S&OP objectives is called forecasting. It allows the implementation of predictive and forecasting models quantifying the evolution of criteria or business variables such as goods and human flows, pricing and product assortments as a function of time.

To successfully model these forecasts, companies must leverage their data and combine it with external data. What data is necessary and useful to successfully learn a forecasting model? What data do retailers, e-tailers, mass-market retailers and manufacturers need to provide in order to make an advanced analysis and an optimal sales forecast possible? Here we take a look at the data needed to build machine learning models capable of monitoring your market or predicting your business and marketing needs.

Sales forecasting in industry, e-commerce and retail

Sales forecasting is one of the applications of forecasting in data science. In order to set up a forecasting model, machine learning specialists use data from the past and the present to predict the evolution of certain variables in the future. The analysis of past trends is one of the mechanisms used to estimate and anticipate these evolutions. Other artificial intelligence techniques are also necessary to process data before the forecast is implemented (data mining, classifications, etc.).

Machine learning and data mining models (clustering, supervised classification, etc.) make it possible to extract recurring patterns hidden within the data stored by your company in order to exploit them in the implementation of predictive models. Starting with a use case, defined with the client company, a player like Verteego can define the data required to meet the client's needs. The data that is useful and essential for modeling a forecast linked to inventory forecasting or pricing strategies includes all the information describing the phenomena and events that can have an impact on these processes.

Thus, the construction of predictive models requires the analysis of historical data specific to the company as well as the cross-referencing of these with variables or data external to your structure, such as meteorological or economic data of the country concerned. The goal behind the choice of data is to ensure that all events that may interfere, influence or modify the course of sales and customer flows within the points of sale are taken into account.

The historical business data needed for forecasting

Verteego advises companies in the selection of data to be used in order to ensure the success of their project, whether it is to support players in the retail, distribution or industrial sectors in the optimization of their S&OP processes.

Internal data defines the first type of data needed to implement pricing strategies for sales, promotions, non-promotional sales and optimized sales. This first category includes historical and internal company data. This data gathers all the information collected in the past about the company's activities. This data can include:

data related to the products or services offered by your company (name, categories, brand, packaging, etc.);
information about the different sales channels and modalities you invest in (points of sale, delivery, drive, click and collect, web application, mobile application, etc.);
characteristics of the points of sale (POS), i.e. their geographical location, size, local events taking place near these points of sale, assortments, etc. ;
information about the sales force of the stores (remuneration, qualification, attendance and visits according to the POS hours);
History of promotions and promotional mechanisms implemented in the past (advertising budgets invested, types of ads, SEA channels, social networks, merchandising mode, catalog displays, etc.).

The goal behind collecting these data tables is to provide a comprehensive and accurate view of your company's product offerings to ensure quality modeling later on. However, collecting the internal data of your organization is not enough. Indeed, the processes and functioning of a business are not impervious to external events (weather, season, politics, economy, vacations, strikes, etc.). Thus, the evolution of the sale of a given product can be explained or correlated to external events.

External data to enrich internal data

There is therefore a second type of data needed to create forecasting models. This information is external to your company. It will provide information on elements or conditions that are external to the functioning of your structure, but which may have an impact on it. This data concerns:

weather conditions;
economic contexts of the targeted geographical area;
competitive information;
factors related to special events such as epidemics (COVID-19).

The cross-referencing of internal and exogenous data feeds the accuracy of the forecasting and provides more complete information about factors that may affect your sales processes or other elements to be optimized.

Forecasting and data importance levels

Not all data used in the creation of machine learning models and artificial intelligence tools are equally important. The level of usefulness or importance depends on the use case or objective set up front for the data analysis project. The data collected can be classified by order of importance: essential, valuable, complementary or useful exogenous.

In the context of sales optimization, for example, priority will be given to details related to the products sold. In this case, sales receipts or invoices provide the chronological information (dates, times) that is essential for forecasting, as well as the necessary details about the products purchased (price, sales channel, promotion or not, range, brand, etc.). The chronological and product characterization data can then be completed by documents related to the marketing repository (promotions planning, offer durations, activities and scheduled events, etc.) of your company.

Finally, additional data about suppliers, the supply chain and the competition will be used to enrich the global view of the sales patterns of your various points of sale, whether they are physical or electronic. Furthermore, it should be noted that the constraints linked to supply chain management (lead times, logistic and production capacity) and storage can also be studied in the context of their optimization.

The use of internal and external data at Verteego

After collecting the various data required for modeling, Verteego proceeds with the preparation and exploitation of the data. For example, in the case of mass distribution and retail, the information collected is introduced into the Verteego Brain tool in order to generate the "Sales Genome" or the DNA of a point of sale. In other words, Verteego Brain makes it possible to create an identity card for a point of sale. This card establishes characteristics such as the number of customers in the outlet in question, what is sold there, the sales methods (drive, click and collect or other), the opening hours, etc.

Other more detailed information concerning the purchase of products will emerge when the Sales Genome is established:

products purchased;
cannibalization effects between products (this data provides information on the products that are preferred and purchased by customers instead of another product);
reasons for purchasing that lead customers to consume a particular product. This information is extracted in particular through the analysis of sales receipts. The analysis of a customer's basket, represented by a receipt, allows us to establish links between products. The co-occurrences of articles within the receipts can inform on the links existing between them and guide the companies in the choice of the assortments of products in the future for example;
the seasonality of products or the time periods and frequencies with which they are purchased by customers in the analyzed outlet.

Artificial intelligence techniques developed by Verteego allow to highlight patterns providing structured and exploitable information for manufacturers and retailers in order to optimize their sales, storage and merchandising processes among others. The modeling of variables to be optimized according to the time dimension requires the exploitation of historical and chronological data of companies wishing to optimize their business and marketing processes. The success of these projects also relies on taking into account parameters that are exogenous to the companies and have an impact on their operations. The collection of these external data provides additional details to consolidate the model and the forecasts it generates.