On Prediction

Suppose that you are a parent of a young child. Every night you give your child a glass of milk with their dinner. When your child is very young, they have a lid on their cup to prevent it from spilling. However, there comes a time when you let them drink without the lid. The absence of a lid presents a possible problem: spilled milk. Initially there is not much you can do to prevent milk from being spilled. However, over time, you begin to notice things that predict when the milk is going to be spilled. For example, certain placements of the cup on the table might make it more likely that the milk is spilled. Similarly, when your child reaches across the table, this also increases the likelihood of spilled milk. The fact that you are able to notice these “risk factors” means that, over time, you will be able to limit the number of times milk is spilled. You begin to move the cup away from troublesome spots before the spill. You institute a rule that the child is not allowed to reach across the table to get something they want. By doing so, the spills become less frequent. You might even get so good at predicting when the milk will spill and preventing it from happening that when it does happen, you and your spouse might argue about it. In fact, with the benefit of hindsight, one of you might say to the other “how did you not see that the milk was going to spill?”

Now suppose that there was an outside observer who studied the spilling of milk at your house. They are tasked with difficult questions: How good are you at successfully predicting when milk is spilled? Were any of your methods to prevent spilling actually successful?

In theory these don’t sound like hard questions. For example, if the observer notices that you are taking preemptive action and the spilling is becoming less frequent, then isn’t this evidence that you are doing a good job at both predicting and preventing spills? Not necessarily. Your child might be maturing and gaining more experience with drinking out of a cup with no lid and therefore less likely to spill their milk. In addition, we would need to know the counterfactual of what would have happened if you had not taken action or created a particular dinner rule for your child. In other words, we need to know whether your child would have spilled the milk if you had not taken the action that you did.

Now, let’s imagine a scenario in which the observer studying your dinner table is naive and just records what happens. Based on their observations, the observer then has to explain why the milk spills. Since the naive observer sees you take action (perhaps even frequently), but also records instances where the milk spills, the observer might come away with the conclusion that you know how to prevent spills (they see you taking such actions), but that you don’t do a good job predicting spills. Their recommendation would be that you need to get better at predicting spills.

As you have certainly realized by now, this post is not meant to be about milk or the weird person observing your dinner habits. It is really about business cycles and countercyclical policy. Naive critics of macroeconomics often point to recessions (especially severe recessions) and say “why didn’t macroeconomists see this coming?” This is an incredibly naive and silly critique in the same way that concluding that you could prevent all of your child’s spills if you were just better at prediction. This view is naive for several reasons. First, we do not have the counterfactual. What would have happened if we had done things differently? It is possible that it might have prevented what we observed, but we need to have a model of how things would have played out differently. It is also possible that there was nothing that we could do or that our actions could have made things even worse. Second, even if we live in a world in which there is some Pareto-improving policy that would have prevented the recession and everyone knows it, this doesn’t mean that we would never have recessions. In fact, in a world of a commonly known Pareto-improving policy, recessions would only occur when they were not predicted. In other words, virtually by definition, recessions would be unpredictable events in that world. To the naive observer, however, they only see the data. They do not have the counterfactuals. Thus, they are likely to conclude that macroeconomists are terrible at their jobs because they never see the recession coming. Put differently, their criticism of macroeconomics would be that macroeconomists fail to predict unpredictable events. That critique is as silly as crying over spilled milk.