When the Driver Stops Driving
Relationships that held for years stop holding, usually gradually. How to notice, what causes it, and why a coefficient should have a review date on it.
A driver relationship that held for three years will stop holding, and it will usually stop gradually rather than suddenly. The coefficient stays in the model, the forecast gets slowly worse, and because the deterioration is spread over quarters nobody attributes it to anything in particular.
The method in “When the Driver Stops Driving” is only as useful as the definitions and source records behind it. A team exploring check the provider’s feature information for remote desktop monitoring software can add timely work and project context, with transparent settings, limited access and a documented correction route around every consequential use.
Every estimated relationship should carry a date and a review interval for exactly this reason.
The assumptions behind “When the Driver Stops Driving” can also be tested against BCG operations resources. Record which guidance applies to the available series and which does not.
Why relationships decay
The audience changes. The channel mix changes. The process changes. Customers learn. A driver that moved volume through one route finds the route closed.
- The audience changes, or learns.
- The channel mix changes.
- The process changes.
- The route the driver worked through is closed.
None of this is unusual; it is what operations do.
The clearest example
Campaign lift falling as send frequency rises. The coefficient was estimated when sends were quarterly and is applied now that they are monthly, and the audience responds to the fourth message of the quarter very differently from the first.
Same driver, same mechanism, a third of the effect.
The second clearest
A billing peak flattening as customers move to direct debit and paperless statements. The dates are unchanged, the peak is half what it was, and the model still carries the old size.
This one is usually visible in the data years before anybody looks.
Detecting decay
Plot the estimated effect by occurrence, in order. Thirty campaigns, a point each. A downward slope is decay; scatter around a flat line is noise.
It takes one chart and it is not in anybody's standard pack.
The alternative: rolling re-estimation
Estimate each driver from the most recent N occurrences rather than from all of them. It tracks decay automatically at the cost of a noisier estimate.
Where occurrences are frequent, this is the better default. Where they are rare, there is no choice but to re-estimate deliberately and watch.
The driver that inverted
Rarer and more damaging. A relationship that reverses — a channel that used to absorb demand now generating it, a change that used to reduce contacts now increasing them.
These are nearly always caused by a known change, which is why the change calendar and the driver register belong next to each other.
The driver that stopped existing
A process retired, a product withdrawn, a communication discontinued. The coefficient remains and contributes zero, which is harmless, or it remains attached to a variable that now means something else, which is not.
An annual read-through of the register against reality catches both.
Review intervals by driver type
Frequent, high-volume drivers — campaigns, billing: annually. Infrequent ones — launches, regulatory events: at each occurrence. Structural ones — channel mix: quarterly, because they move fastest.
Written next to each coefficient, these take the decision out of memory.
What a review actually involves
Re-running the same comparison that produced the original estimate, with the newer data, and comparing. Half a day per driver at most, and for most drivers a morning covers several.
The output is a new number and a new date, or confirmation of the old one.
The coefficient with no date
Which is most of them in most organisations. It cannot be reviewed because nobody knows when it was set or on what data.
Adding a date retrospectively, even an approximate one, is worth doing: "estimated some time before 2023" is more useful than silence.
The model that compensates
A worrying case: a model whose other terms absorb the decay, so overall accuracy holds while the attribution becomes nonsense. The forecast works and the explanations it supports are wrong.
This shows up as drivers whose estimated effects wander between re-fits, and it is a reason to look at the coefficients rather than only at the error.
Decay and the error record together
The two diagnostics complement each other. The error record says the forecast is drifting; the coefficient plot says which driver is causing it.
Neither alone identifies the problem, and most functions keep the first and not the second, which is why drift is usually attributed to the method.
Re-estimation as a scheduled task
Not a project. A standing entry in the quarterly review: which drivers are due, who runs them, what the new numbers were.
Scheduled, it takes a morning a quarter. Unscheduled, it happens after a bad year and is treated as remedial work.
What to tell the reader
Nothing routine. But when a driver is re-estimated and has moved materially, that belongs in the forecast's commentary, because it is usually a fact about the business rather than about the model.
"Campaign lift has fallen from 18% to 9% over three years" is a finding the marketing function will want, and it came out of a forecasting review.