Why are some things easier to forecast than others?

Fore­cast­ers are often met with skep­ti­cism. Almost every time I tell some­one that I work in fore­cast­ing, they say some­thing about fore­cast­ing the stock mar­ket, or fore­cast­ing the weather, usu­ally sug­gest­ing that such fore­casts are hope­lessly inac­cu­rate. In fact, fore­casts of the weather are amaz­ingly accu­rate given the com­plex­ity of the sys­tem, while any­one claim­ing to fore­cast the stock mar­ket deserves skep­ti­cism. So what is the dif­fer­ence between these two types of fore­casts, and can we say any­thing about what can be rea­son­ably be fore­cast and what can’t?

Clearly, some things are eas­ier to fore­cast than oth­ers. The time of the sun­rise tomor­row morn­ing can be fore­cast very pre­cisely. On the other hand, tomorrow’s lotto num­bers can­not be fore­cast with any accu­racy. The pre­dictabil­ity of an event or a quan­tity depends on sev­eral fac­tors including:

  1. how well we under­stand the fac­tors that con­tribute to it;
  2. how much data are available;
  3. whether the fore­casts can affect the thing we are try­ing to forecast.

For exam­ple, fore­casts of elec­tric­ity demand can be highly accu­rate because all three con­di­tions are usu­ally sat­is­fied. We have a good idea on the con­tribut­ing fac­tors:  elec­tric­ity demand is largely dri­ven by tem­per­a­tures, with smaller effects for cal­en­dar vari­a­tion such as hol­i­days, and eco­nomic con­di­tions. Pro­vided there is a suf­fi­cient his­tory of data on elec­tric­ity demand and weather con­di­tions, and we have the skills to develop a good model link­ing elec­tric­ity demand and the key dri­ver vari­ables, the fore­casts can be remark­ably accurate.

On the other hand, when fore­cast­ing cur­rency exchange rates, only one of the con­di­tions is sat­is­fied: there is plenty of avail­able data. How­ever, we have a very lim­ited under­stand­ing of the fac­tors that affect exchange rates, and the fore­casts of the exchange rate have a direct effect on the rates them­selves. If there are well-​​publicized fore­casts that the exchange rate will increase, then peo­ple will imme­di­ately adjust the price they are will­ing to pay and so the fore­casts are self-​​fulfilling. In a sense the exchange rates become their own fore­casts. This is an exam­ple of the effi­cient mar­ket hypoth­e­sis. Con­se­quently, fore­cast­ing whether the exchange rate will rise or fall tomor­row is about as pre­dictable as fore­cast­ing whether a tossed coin will come down as a head or a tail. In both sit­u­a­tions, you will be cor­rect about 50% of the time what­ever you fore­cast. In sit­u­a­tions like this, fore­cast­ers need to be aware of their own lim­i­ta­tions, and not claim more than is possible.

Often in fore­cast­ing, a key step is know­ing when some­thing can be fore­cast accu­rately, and when fore­casts are no bet­ter than toss­ing a coin. Good fore­casts cap­ture the gen­uine pat­terns and rela­tion­ships which exist in the his­tor­i­cal data, but do not repli­cate past events that will not occur again.

Many peo­ple wrongly assume that fore­casts are not pos­si­ble in a chang­ing envi­ron­ment. Every envi­ron­ment is chang­ing, and a good fore­cast­ing model cap­tures the way things are chang­ing. Fore­casts rarely assume that the envi­ron­ment is unchang­ing. What is nor­mally assumed is that the way the envi­ron­ment is chang­ing will con­tinue into the future. That is, that a highly volatile envi­ron­ment will con­tinue to be highly volatile; a busi­ness with fluc­tu­at­ing sales will con­tinue to have fluc­tu­at­ing sales; and an econ­omy that has gone through booms and busts will con­tinue to go through booms and busts. A fore­cast­ing model is intended to cap­ture the way things move, not just where things are. As Abra­ham Lin­coln said “If we could first know where we are and whither we are tend­ing, we could bet­ter judge what to do and how to do it”.

 


This is an edited ver­sion of Sec­tion 1­/​1 from my new book Fore­cast­ing: prin­ci­ples and prac­tice.


Related Posts:


  • http://pulse.yahoo.com/_3NOMIDA6UZ6LNXUGCTEZSWBQOY Bug­gy­Fun­Bunny

    You should start with the Wike­pe­dia arti­cle on effi­cient mar­kets. Fore­casts, of for­eign exchange in par­tic­u­lar, are gen­er­ally pro­pri­etary prog­nos­ti­ca­tions for the privy priv­i­leged. IOW, sequestered infor­ma­tion, which is the antithe­sis of the EM hypothesis.

    In a nut­shell, what makes fore­cast­ing dif­fi­cult is what kind of process you’re attempt­ing to fore­cast. Weather does well since it is purely mech­a­nis­tic (although with so many para­me­ters that super­com­put­ers are needed to track it all), while pre­dict­ing any part of eco­nomic activ­ity is fraught with “Dan­ger, Will Robinson!!”

    The Great Reces­sion (and one may be brew­ing in Oz?); the fore­cast­ers could have got­ten it right if they’d been bet­ter econ­o­mists, who under­stood the fun­da­men­tal ratios in the hous­ing mar­ket ( median price /​ median income, is one key). And knew that Greenspan had crashed inter­est rates in 2002. And that China, due to its macro poli­cies, gen­er­ated out­lier amounts of moolah. And that the wealthy (of all loca­tions) demanded bet­ter “risk free returns” than US debt was then offer­ing (and still does). And that US mort­gage com­pa­nies (not so much banks) were free to make any qual­i­fy­ing rules they felt like. And that Banksters (Spain’s prob­lem, too) were more than happy to play sausage maker with those loans. The list goes on.

    IOW, pol­icy (mak­ing it gives one a real leg up, know­ing it before most oth­ers, a smaller one) trumps data every time. And, no, I’m not in the Black Swan camp. There are no co-​​incidences or Black Swans; just insuf­fi­cient data.

  • Pingback: 왜 어떤 것은 더 예측이 쉬운가? « NewsPeppermint

  • Pingback: Tim Harford on forecasting | Hyndsight