Anti-Moneyball. How Poor Decision-Making Can Fail Organizations

The Loser’s Curse: Decision Making and Market Efficiency in the National Football League Draft

In 2013 Richard Thaler, the Professor at the University of Chicago Booth School of Business together with the Professor from The Wharton School of the University of Pennsylvania Cade Massey published a very interesting paper - The Loser’s Curse: Decision Making and Market Efficiency in the National Football League Draft.

In this article authors tells about the behavioral aspects and the importance of good decision-making during the NFL draft:

"A question of increasing interest to researchers in a variety of fields is whether the biases found in judgment and decision-making research remain present in contexts in which experienced participants face strong economic incentives. To investigate this question, we analyze the decision making of National Football League teams during their annual player draft. This is a domain in which monetary stakes are exceedingly high and the opportunities for learning are rich. It is also a domain in which multiple psychological factors suggest that teams may overvalue the chance to pick early in the draft. Using archival data on draft-day trades, player performance, and compensation, we compare the market value of draft picks with the surplus value to teams provided by the drafted players. We find that top draft picks are significantly overvalued in a manner that is inconsistent with rational expectations and efficient markets, and consistent with psychological research."

In addition, authors previously worked on this topic, and in 2005 their working paper Overconfidence vs. Market Efficiency in the National Football League was published by the National Bureau of Economic Research.

In their “The Loser’s Curse” paper Professors Thaler and Massey investigated how did rational expectations and market efficiency play out in the NFL labor market.

“To determine whether the market values of picks are “correct,” we compare them to the surplus value (to the team) of the players chosen with the draft picks. We define surplus value as the player’s performance value—estimated from the labor market for NFL veterans—less his compensation."

According to the authors, their findings

"strongly reject the hypothesis of market efficiency. Although the market prices of picks decline sharply initially, we find surplus value of the picks during the first round actually increases throughout most of the round: the player selected with the final pick in the first round, on average, produces more surplus to his team than the first pick!"

So, what did Thaler and Massey do?

• For the initial step in their analysis, they used a data set of 407 draft-day trades to estimate the market value of draft picks (trades of these draft picks from 1983–2008)

• Authors asked whether the highest picks were too expensive, as the relevant psychology predicts, and used a non-parametric approach, comparing the benefit of using a pick to the opportunity cost of foregone trades

• Finally, they performed a more detailed cost-benefit analysis of each player selected in the draft, by calculating the surplus value that the player provided to the team, namely, the performance value (estimated by the price of an equivalent veteran player) minus the salary paid

In particular, authors predicted that

“teams will overvalue the right to choose early in the draft. Specifically, we believe teams will systematically pay too much for the right to draft one player over another. This will be reflected in the relative price for draft picks as observed in draft-day trades.”

The results of the research were surprising even for the authors of the study.

The authors highlighted the following:

“A striking feature of these data is how steep the curve is. The drop in value from the 1st pick to the 10th is roughly 50%, and values fall another 50% from there to the end of the first round. As we report in the following section, compensation costs follow a very similar pattern. Although the curve is not as steep as it used to be, this flattening has slowed over time. In an efficient market the curve’s steepness would imply both that player performance falls sharply at the top of the draft, and that teams are highly skilled in their ability to identify these performance differences.”

Moreover, the authors noted that:

“Another notable feature is the remarkably high discount rate, which we estimate to be 136% per year. Although this finding is not the focus of the paper, it is clear that teams who “borrow” picks on these terms are displaying highly impatient behavior."

Besides Thaler and Massey highlighted that teams were not doing a very good job, estimating the likelihood that a player is better than the next player chosen at his position:

"The very steep curve we document in this section suggests that teams believe they have the ability to distinguish great players from the merely good. Before moving to full cost-benefit analyses, let us consider a simple question: What is the likelihood that a player is better than the next player chosen at his position (e.g., linebacker) by some reasonable measure of performance, such as games started in his first five seasons? After all, this is the question teams face as they decide whether to trade up to acquire a specific player. The answer is 52%. Across all rounds, all positions, all years, the chance that a player proves to be better than the next-best alternative is only slightly better than a coin flip."

And that is exactly what the “Moneyball” was about - the inability of “industry experts” without a proper data analysis to distinguish and select the most valuable players on the market due to own biases.

In addition, the authors explored the opportunity costs and conducted the cost-benefit analysis of the NFL draft.

First, Thaler and Massey evaluated that

“the benefit of using a draft pick relative to the opportunity cost of trading it for two lesser picks”. They found overwhelming evidence that “that a team would do better in the draft by trading down (this gain to be reliably positive for 31 of the 32 draft-pick positions in the first round.” Moreover, “For 74% of the trades, a team would have acquired more starts by trading down than by using a pick.” Hence, teams would be much better off, exchanging their current first round pick with two future picks later in the draft.

Second, authors identified and later picks are more beneficial for the team financially.

“Consequently, surplus value increases at the top of the order, rising to its maximum of approximately $1,200,000 near the beginning of the second round before declining through the rest of the draft. That treasured first pick in the draft is, according to this analysis, actually the least valuable pick in the first round! To be clear, the player taken with the first pick does have the highest expected performance (that is, the performance value curve is monotonically decreasing), but he also has the highest salary, and in terms of performance per dollar, is less valuable than most players taken in the second round.”

The last pick in the first round is worth only 25% of the first pick even though the last pick will command a much smaller salary than the first pick. These simple facts are incontrovertible. In a rational market, such high prices would forecast high returns; in this context, stellar performance on the field. And, teams do show skill in selecting players: Using any performance measure, the players taken at the top of the draft perform better than those taken later. In fact, performance declines steadily throughout the draft. Still, performance does not decline steeply enough to be consistent with the very high prices of top picks. Indeed, we find that the expected surplus to the team declines throughout the first round. The first pick, in fact, has an expected surplus lower than any pick in the second round and is riskier as well. Furthermore, the risk associated with very high picks is mostly on the downside. Because top picks are paid so much, there is little room for a player to greatly exceed expectations, but when top picks turn out to be complete busts, tens of millions of dollars are wasted.

As a result, Thaler and Massey concluded the following:

“A conclusion from our analysis is that teams should trade down, not up. A possible objection to this conclusion is that when teams trade up, they might have a special need at the position and/or believe they have particularly good information about the acquired player. … Our modest claim in this paper is that the owners and managers of National Football League teams are subject to the same biased judgments found in countless other domains. Furthermore, market forces have not been strong enough to overcome these human failings. The task of picking players, as we have described in this paper, is an extremely difficult one, much more difficult than the tasks psychologists typically pose to their subjects.”

The final thoughts of the authors:

"Our findings are strikingly strong. Rather than a treasure, the right to pick first appears to be a curse. If picks are valued by the surplus they produce, then the first pick in the first round is the worst pick in the round, not the best. In paying a steep price to trade up, teams are paying a lot to acquire a pick that is worth less than the ones they are giving up. The implications of this study extend beyond the gridiron. At its most general, these findings stand as a reminder that decision makers often know less than they think they know. This lesson has been implicated in disaster after disaster, from financial markets to international affairs. Closer to the topic at hand, football players are surely not the only employees whose future performance is difficult to predict. In fact, football teams almost certainly are in a better position to predict performance than most employers choosing workers, whether newly minted MBAs or the next CEO. In our judgment, there is little reason to think that the market for CEOs is more efficient than the market for football players. The problem is not that future performance is difficult to predict, but that decision makers do not appreciate how difficult it is."

Overconfidence vs. Market Efficiency in the National Football

There are also several more interesting points in the NBER working paper, written by authors in 2005. This paper used a smaller dataset, have similar conclusions, and provide some additional interesting ideas.

For instance, analyzing the data, Thaler and Massey found out the chance to be failed in the league for first round picks are similar to the chance to play in a Super Bowl:

“For example, over their first five years, players drafted in the first round spend about as many seasons out of the league (8%) or not starting a single game (8%) as in the Pro Bowl (9%).”

Authors also refer to Michael Lewis:

As Michael Lewis, author of Moneyball, said of another sport, “If professional baseball players, whose achievements are endlessly watched, discussed and analyzed by tens of millions of people, can be radically misvalued, who can’t be? If such a putatively meritocratic culture as professional baseball can be so sloppy and inefficient, what can’t be?”

Finally, Thaler and Massey provide some specific examples:

“We began this study with the strong intuition that teams were putting too high a value on choosing early in the draft. We thought it crazy for the Giants to give up so many picks for the opportunity to move up from the fourth pick to the first one (regardless of which player they chose). But we concede that we did not expect the findings to be as strong as those we report. Rather than a treasure, the right to pick first appears to be a curse. If picks are valued by the surplus they produce, then the first pick in the first round is the worst pick in the round, not the best! In paying a steep price to trade up, teams seem to be getting the sign wrong! We have done numerous “reality checks” to see whether these surprising conclusions are robust, and every analysis gives qualitatively similar results. So, suppose our analyses are taken at face value. Can they be right? This is a big market, after all, with franchises worth perhaps $1 billion or more. We think that while our results are surprising, they are plausible. We suspect that some teams have not fully come to grips with the implications of the salary cap, a relatively new innovation. Buying expensive players, even if they turn out to be great performers, imposes opportunity costs elsewhere on the roster. Spending $10 million on a star quarterback instead of $5 million on a journeyman implies having $5 million less to spend on offensive linemen to block or linebackers to tackle. Some of the successful franchises seem to understand these concepts, most notably the New England Patriots, but others do not. Whether because they are smart about these ideas or others, the Patriots have been doing well recently, and so have not had high draft picks to use. We can only speculate about whether they would trade down if they somehow ended up with one of the earliest and most overvalued picks. But notice that if a few teams do learn and have winning records, there is no market action they can take to make the implied value of draft picks rational. Indeed, the irony of our results is that the supposed benefit bestowed on the worst team in the league, the right to pick first in the draft, is really not a benefit at all, unless the team trades it away. The first pick in the draft is the loser’s curse."