The Outlier That Was a Real Day
When to remove an extreme value from a demand series and when removing it destroys the thing you needed to know, with the three questions that decide.
An extreme value should be removed from a series when it was caused by something that will not recur, and kept when it was caused by something that will. Nothing about the size of the value tells you which, which is why outlier removal by threshold is a bad idea however convenient.
The series discussed in “The Outlier That Was a Real Day” is easier to interpret when changes in work capture are visible rather than silently folded into the history. Used for mouse jiggler detection, a practical approach to mouse jiggler detection can provide consistent time and project evidence, provided corrections, category changes and missing periods remain traceable.
The day that was four times normal might be a data error, or it might be the kind of day the staffing plan most needs to anticipate.
As a separate benchmark for “The Outlier That Was a Real Day”, see Bain operations insights. Keep source data, model, override and resulting decision visible as distinct steps.
The three questions
What caused it. Can that cause happen again. And would the organisation want to be staffed for it if it did.
- What caused it?
- Can that cause happen again?
- Would we want to be staffed for it if it did?
All three are answerable in a few minutes by asking somebody who was there, and that conversation is the whole method.
Causes that will not recur
A duplicate load in the data. A one-off migration of historical records into the live queue. A single customer's bulk action. A test that ran in production.
These are not demand and removing them is correct.
Causes that will recur
A system outage elsewhere that displaced contacts here. A regulatory deadline. A failed release. Severe weather. A competitor's failure.
These are demand, they will happen again, and removing them produces a forecast that is wrong exactly when it matters.
The compromise that works
Keep the day in the series, label the cause, and model it as an event rather than as baseline. The baseline forecast is then clean and the event is a known, separately estimated addition.
This is more work than deleting and it is the difference between a forecast that plans for bad days and one that is surprised by them.
Automatic outlier removal
Available in most tools, usually as a standard deviation threshold. It is appropriate for data errors in high-frequency series and inappropriate for daily demand, where the extremes carry the information about risk.
Where it is on by default, it should be turned off and replaced by a list.
The list
Every value flagged as extreme, with its date, its cause, and the decision taken. Fifteen rows a year in a typical operation.
Reviewed once, it usually shows that a third of what was being removed was real.
The outlier at the top versus the bottom
Low extremes are more often data problems — outages, partial days, closures recorded wrongly. High extremes are more often real.
That asymmetry is worth knowing because the automatic methods treat them identically.
What removal does to the band
It narrows it, which is the attraction and the danger. A series with its bad days removed produces a confident forecast that has never seen a bad day.
The resulting staffing plan is efficient in ordinary weeks and fails in precisely the weeks that get noticed.
Planning for the tail
Where extremes are real and recurrent, the useful output is not a smoother forecast but a second number: how often a day exceeds the staffing level, and what happens when it does.
That is a different question from the forecast and it is answered from the same series, by looking at the distribution rather than the average.
The run of unusual days
Harder than a single outlier. Six weeks of elevated volume is either a new level, a long event, or the start of a trend, and the three want opposite treatments.
The only reliable discriminator is the cause, which means asking rather than testing. Waiting one more period before deciding is usually cheaper than deciding wrongly.
Documenting the decision
Date, value, cause, decision, who decided. Five columns in the same record that holds definition changes and gap fills.
The record is what lets a future planner re-examine a judgement instead of re-making it from nothing.
The outlier in the forecast rather than the history
A different case worth naming: a forecast that produces an implausible value because a driver was entered wrongly or a seasonal factor misapplied.
A sanity check before publication — is this within some multiple of recent comparable periods — catches these, and they are much more embarrassing than an honest miss.
How extremes are distributed
Worth looking at once: in most demand series the extremes are not symmetric and not rare. A series with ten per cent of days outside two standard deviations is normal, not broken.
That observation alone retires most automatic threshold rules.
The day that was extreme for several reasons
The hardest to classify: a release, a billing date and bad weather in the same week. Attributing the whole excess to one of them overstates it and corrupts that driver's estimate.
Where causes overlap, the honest entry is "multiple, not separable", and the day is excluded from driver estimation rather than assigned.
The question that settles most cases
"If this happened again next month, would we want the forecast to have seen it coming?" Asked of the operations manager, it produces a clear answer in almost every case, and it is the right question because it is about the decision rather than about the statistics.