Why Zero Sales Come In (At Least) Two Kinds
Summary: In retail forecasting, zero-sales events require special attention when training and applying demand models. It is difficult to find out ex post whether a zero-sales-event truly witnesses vanishing demand on a given day (as in “nobody took that product from the shelf”), or whether…
Forecasting Evaluation Pitfalls
In Part 1 of this blog post, we introduced censored sales probability distributions. Let’s now get our hands dirty and see what finite capacity means in practice. We start by pointing out subtle pitfalls that one might inadvertently fall into in order to then share…
Why We Expect To Sell Less Than We Predict
How to account for finite capacity in demand forecasting training and evaluation Summary In retail forecasting, the quantity of interest is the customers’ demand for a certain product, e.g., how many baskets of strawberries are being requested. In practice, one observes a slightly but significantly…
Calibration and Sharpness: The Two Independent Aspects of Forecast Quality
What Is a Good Forecast? Forecasts are like friends: Trust is the most important factor (you don’t ever want your friends to lie to you), but among your trustable friends, you prefer meeting those that tell you the most interesting stories. What do I mean…
How To Fix Mean Absolute Error, Part 2
Part 2: How the Ranked Probability Score reconciles statisticians with practitioners In part 1, we challenged the usual way to evaluate Mean Absolute Error by just taking the difference between the predicted mean and the observed outcome. We found that it is necessary to use…
How To Fix Mean Absolute Error, Part 1
The deceiving surprises that Mean Absolute Error hides from us Executive summary Mean Absolute Error is the first choice for practitioners when it comes to evaluate their model, owing to its simple definition and its intuitive business relevance. The evaluation metric Ranked Probability Score, by…
Mean Absolute Percentage Error (MAPE) Has Served Its Duty and Should Now Retire
Executive summary According to Gartner (2018 Gartner Sales & Operations Planning Success Survey), the most popular evaluation metric for forecasts in Sales and Operations Planning is Mean Absolute Percentage Error (MAPE). This needs to change. Modern forecasts concern small quantities on a disaggregated level such…
Forecasting Few Is Different, Part 2
This is a two-part story by Malte Tichy about dealing with sales forecasts that concern fast-moving as well as slow-moving items. Find Part 1 here. How precise can a sales forecast become? The mechanism of fluctuation cancellation ensures that granular low-scale-forecasts are more imprecise, noisy…
Forecasting Few Is Different, Part 1
When dealing with sales forecasts that concern fast-moving as well as slow-moving items, we must account for the non-proportional scaling of relative forecast uncertainty with selling rates, which largely determines the achievable level of precision. Executive summary For the same forecast quality, predictions for slow-moving…
You Should Not Always Have Known Better: Understand and Avoid the Hindsight Selection Bias in Probabilistic Forecast Evaluation
This blog post on forecast evaluation discusses common pitfalls and challenges that arise when evaluating probabilistic forecasts. Our aim is to equip Blue Yonder’s customers and anyone else interested with the knowledge required to setup and interpret evaluation metrics for reliably judging forecast quality. The…