What do you do when your insights challenge prevailing beliefs? That’s the question we faced at IBM in 2007 when a brand new forecasting tool we’d developed started spitting out projections that conflicted with our other forecasts.
For our paper, “Marketing Science: From Descriptive to Prescriptive,” we looked at IBM’s own experiences to learn about the challenges and advantages that relying on data can create.
Large companies, like IBM, can really struggle with pulling together complete and relevant data to create accurate forecasts. We initially developed the new forecasting tool when IBM missed an earnings target in 2005. After studying the forecasting technology we had been relying on, the Market Development and Insights team discovered that its focus on the longer term had failed to pick up on a shift in the market. That meant we had to improve the accuracy of the longer term view with some sort of early warning system to flag abrupt changes in market direction.
The new tool was designed to track over 2,000 market indicators in the G7 and BRIC countries, including oil prices, key manufacture goods and transportation, on a monthly basis. It uses correlation techniques, regression testing and principal components analysis to identify the best indicators, given the current market climate. Then it combines them to produce a forecast of the market for the next three quarters.
By 2007, our early warning system was ready to roll out to handle short term forecasting and supplement the long view.
There was just one issue: No one knew whether to trust it. Because it was forecasting that black clouds were gathering on the horizon. But our long view and all the third-party sources we used said the sky was blue.
IBM’s top management was understandably dubious of the predictions of an unproven new forecasting tool. So we decided to take the time to track economic data on our own very carefully for two quarters and compare it with forecasted findings.
The result? The alert tool was more accurate. In fact, it was so accurate that we were able to predict the recent global recession two quarters before third-party sources detected it. This six-month lead gave us time to restructure our cost base and realign our investments.
We now use a short term approach to predict the state of the hardware, software and services market by country for the next six months. We’ve also closed the loop by feeding the projections from this into the long term tools to continuously refine our long-term forecasts and monitor downside risks as well as upside opportunities.
Of course, no forecasting tool will be 100% spot on all the time. But as companies battle test and tweak the array of tools at their disposal, they get closer all the time.
Related posts:
