Tonight, the NFL season kicks off with a rematch of one of last year’s AFC Divisional Round games as the Baltimore Ravens travel to Denver to play the Broncos. Who has the edge in a matchup that pits the reigning Super Bowl Champions against a team that went 13-3 in 2012?
As fans, we watch the games week to week, but we also consume the derivative product of prognostication. Our anxieties are calmed when the NFL expert or former All-Pro Quarterback turned NFL Analyst/Nutrisystem spokesperson asserts that our team will win on Sunday. But should they be?
Over the past few years there have been efforts to add accountability to punditry. Most notably, Pundittracker.com monitors analysts’ predictions on topics ranging from politics and finance to the NFL, and provides a track record for their past picks.
To put analysts’ accuracy into perspective, we can compare their success against the hit rates for Accuscore, a simulation-based sports betting system, or use the approach of simply choosing whoever oddsmakers in Las Vegas are favoring in betting lines.
The 2012-13 campaign was a good one for the pundits, with half of the 22 analyzed equaling or besting the Vegas and Accuscore hit rates. That said, Vegas odds were uncharacteristically low, at 65 percent, down from their average of 66.8 percent over the past ten seasons. Yahoo!’s Mike Silver was the only analyst to beat the two benchmarks in the 2011-’12 and 2012-’13 NFL seasons.
Even more interesting is that for the 2011-’12 season, the aggregate majority opinion of Yahoo! users — a collection of average fans — outperformed every analyst and benchmark with a hit rate of 68.1 percent. ESPN users weren’t far behind, correctly picking 66.5 percent of games that season. Data for ESPN and Yahoo! users was unavailable for the 2012-’13 season.
While overall accuracy is important, it would be an oversight to ignore the ‘boldness’ of these analysts’ predictions. An analyst that simply mirrors the Vegas odds isn’t adding any value with his opinion.
General hit-rate shows overall accuracy, but the notion of $1 Bet Yield — based on placing $1 for every pundit’s prediction considering the Vegas money line odds — is another source of determining an expert’s value. The money line is set to only marginally benefit picking a favorite, while offering a huge reward for selecting an underdog. Because of this, it creates a barometer for monitoring a pundit’s strength in correctly calling unpopular picks.
For example, playing on the road against the Broncos, the Ravens might not seem like a very appealing pick to most. The game has been quoted around +270/-300, meaning placing $100 on Denver earns you only $33.33, but placing the same amount on Baltimore rewards $270.
For 2011-2012, this analysis puts Silver on top of the field again and shows six pundits turning a positive return, while 10 take losses on the year. Even adjusted for riskiness, out of those 16 pundits, only four outperformed the Yahoo! user.
There are a few reasons why the Vegas odds and the pooled fan opinions have performed well against the pundits:
1) Money – The better analysis is always going to come from someone that has some skin in the game. The Vegas money lines and spreads must accurately reflect the matchup because there is an inconceivable amount of money being placed based on the odds, and the industry needs balanced action on both sides of each bet.
2) The pressure of punditry – Analysts are encouraged to make particularly unique predictions to separate themselves from the rest of the field. There isn’t anything sexy or insightful about always picking the favorite. This pressure to find the hidden underdog every week drives down many pundits’ overall hit rates – consider how searching for a Cinderella team can ruin a March Madness bracket.
The Yahoo! and ESPN users fared well because the majority opinion of a large group is likely to lean more conservative. Generally, their picks will only deviate from the Vegas implied favorite in games where the spread is small. Over the past decade, spreads within a field goal have correctly indicated the winner 54 percent of the time, while teams favored by more than three have won 73.5 percent of their games. Without the individual picks, it is impossible to do attribution analysis, but it’s overwhelmingly likely that fans have been slightly better at picking these ‘toss-up’ games.
3) Viewership/readership – Punditry also has an entertainment value. Simply put, analysts are filling air time or filling the void in the news cycle in the time leading up to the games. The pre-game and half-time shows exist so that networks can monetize the games beyond their 60 minutes of regulation. Controversial picks lend themselves to on-air debates likely to rattle the cages of viewers at home and keep them glued to the television.
The takeaway? It is difficult for an analyst to consistently outperform the conservative approach of following the Vegas odds. Mike Silver seems to be the best in the business right now, but if you can’t reach him, consult the money lines and poll a few friends.
— Dylan Lewis can be reached at [email protected]
