So it seems last week’s column understandably ticked off a few analysts who objected to the idea that to trust an analytical process to the very end might mean casting aside some of the ideas that are synonymous with what we love most about sport—all powerful managers, the real influence of random variation, etc.
While I wrote that these two domains are irreconcilable, I did not mean that to imply that they can’t also be held by one and the same person, but at different times and in different contexts, to great effect.
Interestingly, my article came out only a few days before the New Yorker’s Louis Menand managed to roughly make this same, crucial point in discussing L. Jon Wertheim and Sam Sommers’ book, This is Your Brain on Sports. Menand writes on fan favouritism and the failure of club supporters to think objectively when it comes to player morals, like those of Peyton Manning:
We don’t think we need to do this, because sports is one of those consequence-free zones in life in which a double standard is acceptable. Armchair political debate is another. Hillary Clinton voted for the Iraq War? An honest mistake, made by many. Bernie Sanders voted against gun regulation? Disqualified to be President! But, once consequences for us or for people we care about come into the picture, we strive to assess candidates objectively. We may be—we undoubtedly are—still limited by blind spots, but we are not flagrantly prejudiced. To be a fan is to make a point of being unreasonable. Sports is a vacation from prudence.
Is hating on Peyton more “rooted in basic human psychology” than appreciating his case with judicious disinterest? People toggle effortlessly between these behaviors every day, because people are astute readers of context. We know the difference between sitting in a bar and sitting in a jury box. We condemn bias when it matters just as instinctively as we root for the home team when it doesn’t.
I am inclined to agree, except that not everyone is an “astute reader of context,” including some of the people in charge of running football clubs. One needs only to recall last week when West Ham owner David Sullivan essentially blamed Manchester United for their coach being pelted with objects by Hammers supporters because they arrived late. In a moment that certainly called for “judicious disinterest,” Sullivan was incapable of it. And if you think politics is any different, see any news item ever involving Donald Trump.
I would also disagree with Menand that this context switching is “effortless.” Just as there are Sullivans within clubs incapable of thinking objectively, the sports analytics world has seen several instances where good analysis was marred by narrative-driven conclusions or explanations grafted on to appease our deeply ingrained sporting prejudices.
Think, for example, of the strong reluctance of some analysts to countenance the idea that a team vastly over-performing against metrics like Expected Goals might really be on a long streak of good fortune, much like an observer may not buy that 11 heads in a row in a coin toss is possible unless the coin is weighted. Sure, there may be some as-yet hidden variable that, once discovered, will explain why a team might go five months straight scoring far more goals than their xG totals might explain. As I’ve said before, that’s what the tape is for. But if an experienced PA can’t find an obvious pattern to explain a long, unlikely scoring run, well, prepare for the #vardyparty to come to an end.
This is at the centre of something I briefly hinted at last week and now want to explore further: while the Burley/Marcotti domain dichotomy in sports is real, knowing when to switch between these modes of understanding the game is itself a vital skill.
Statistics vs Calculus
In discussing the Marcotti/Burley Expected Goals showdown—in particular Craig Burley’s view that “results are all that matter,” which is another way of saying good managers must always find a way to win at all costs or face the consequences—I wrote this:
Craig Burley’s perspective, while obtuse, is a potentially useful mental model, in the same way xGs are part of a useful mental model (pardon the Buffettisms today). It may be better for club managers to speak like Burley and think like Marcotti, lest players don’t strive as hard, comforted by the knowledge that superior performance metrics will let them off the hook for an L.
Another way of understanding these two mental models comes from the admittedly annoying and self-aggrandizing world of tech startups. Peter Thiel’s book Zero to One makes an interesting (if scientifically inaccurate) point about how effective startup companies should view the world not through the hazy, probabilistic lens of statistics, but the crisp, deterministic lens of calculus:
When you send a rocket to the moon, you have to calculate precisely where it is at all times. It’s not like some iterative startup where you launch the rocket and figure things out step by step. Do you make it to the moon? To Jupiter? Do you just get lost in space? There were lots of companies in the 90s that had launch parties but no landing parties.
But the indeterminate future is somehow one in which probability and statistics are the dominant modality for making sense of the world. Bell curves and random walks define what the future is going to look like. The standard pedagogical argument is that high schools should get rid of calculus and replace it with statistics, which is really important and actually useful. There has been a powerful shift toward the idea that statistical ways of thinking are going to drive the future.
To be clear, Thiel’s book is aimed squarely at managers and executives, for whom probabilistic thinking offers an endless fountain of potential justifications and excuses —”our demographics changed, we did our due diligence and followed our leading and lagging indicators but the market shifted, we were almost certainly going to go bankrupt so it made sense to wind down sooner than face higher costs down the line.”
Ben Horowitz, who quotes Thiel in his memoir The Hard Thing About Hard Things, also “believes in calculus”.
Startup CEOs should not play the odds. When you are building a company, you must believe there is an answer and you cannot pay attention to your odds of finding it. You just have to find it. It matters not whether your chances are nine in ten or one in a thousand; your task is the same.
I don’t know but I’m guessing Thiel or Horowitz believe in sales forecasting or market research, or collecting and analysing data to make optimal decisions for the company. A deterministic view isn’t about blind faith or that a little hard work can accomplish literally anything, like selling a crappy product to a market that doesn’t want it.
Rather, it’s about avoiding the tendency to benchmark goals against an amorphous average. It’s about avoiding using probability as a crutch, an excuse not to strive harder for the unknown unknown. It’s about believing that no matter the circumstances, managers and directors always have a choice, another move.
Note that neither a deterministic/probabilistic view is necessarily truer than the other. Their ability to describe the world accurately is less important than knowing which is more useful in which context.
When it comes to football, though most of us know the pitfalls of thinking deterministically in a probabilistic context — sacking a good manager for an unlucky run of results, overpaying for young stars with no data and on the basis of four live matches — we’re less attuned to the potential pitfalls of the reverse:
- A club analyst points out that a poor team performance over a run of matches is down to bad luck, with no additional discussion on whether the benchmark metric itself – perhaps it’s xGD – is ideal based on the ambitions of the club, how much it has spent in the transfer market etc.
- A recruitment process uses good predictive metrics and solid Bayesian reasoning in identifying appropriate transfer market targets. When targets fail to match performance standards, the club agrees it’s ‘random variation’ and writes off the loss with no other follow up actions.
- A club manager well-versed in the influence of luck single games or the home and away effect feels less pressure to strive to influence the outcome in a particular match.
- A club works diligently to eliminate bias in their academy selection process and runs first team quotas for homegrown players, only to see performances crater, at which points it cancels the youth first policy.
So how does a club avoid these kinds of traps?
Football and the Via Negativa
Back in early March of this year, North Yard Analytics’ Dan Altman wrote a nice piece in defense of stats analysis in soccer. In noting that clubs were silently adopting analytics without, in his words, forcing “analytics down its fans’ throats with marketing, in the media, or on game day,” Altman identified their basic purpose:
Analytics in any business are not about winning an ideological war with stakeholders; they’re about making better decisions. It’s hard to see how this could be a bad thing in soccer/football.
How many clubs have paid millions for a player who turned out to be a flop? How many have overspent only to collapse in debt? How many have been blindsided by relegation and tumbled down the pyramid, like once-great Portsmouth? These are wrenching moments for fans. Analytics can’t stop them from happening altogether, but analytics can reduce their frequency.
It is both telling and interesting that Altman points not to any great analytics triumph or program, like the ability to recruit unknown superstars or find the ideal “winning tactic”, but rather what good data analysis can help avoid — chaos, uncertainty, waste.
The acerbic quant and probability theorist Nassim Nicholas Taleb has, in the past, hailed the virtue of what he calls the “via negativa,” a concept derived from a school of thought in theology which held the best way to describe god is to define what she is not. It is a way of being in which progress is not about the habits you embrace, but the mistakes you avoid. By focusing on “removing the bad,” says Taleb, you remove “…the side effects and branching chains of unintended consequences – hence robust.”
He elucidates on this point in his book Antifragile:
In practice it is the negative that’s used by the pros, those selected by evolution: chess grandmasters usually win by not losing; people become rich by not going bust (particularly when others do); religions are mostly about interdicts; the learning of life is about what to avoid. You reduce most of your personal risks of accident thanks to a small number of measures.
Imagine an analyst who doesn’t pipe up much at meetings, except when it appears the powers that be are on the verge of a major mistake. Perhaps the board of directors has lined up a shortlist of managerial candidates. If the analyst suddenly speaks up to tell the board, “Under no circumstances can you hire these two interviewees,” their voice will be louder, if only for the influence of loss aversion. Of course, ideally the club would want a thorough succession plan in place whose sole purpose is to avoid bad managers, but the power to offer a strong no may take you farther than one in which the aim is to give an emphatic yes.
In fact, we already have a potential example in the works, as Manchester United are reportedly seeking a scouting ‘auditor’ whose job is to presumably to carefully review scouting recommendations from abroad.
The benefit of a ‘via negativa’ approach is that it does not view the world through a particular lens or ideology, whether deterministic or probabilistic. It does not write off losses or mistakes as “outliers” as part of a more comprehensive plan, but makes the losses themselves the focus. It provides a natural progression. How do we as a club ensure a higher success rate in recruitment? With better forecasting and better predictive metrics. What if signings still fail? Then we look at our coaching and acclimation process. What if players still fail to hit targets? Then we interview them, ask about problems in their personal life, communication issues with the manager…and if it truly is an outlier and a bad fit, we cut our losses and move on.
A ‘via negativa’ approach eschews projects and visions and goes instead for problem solving. It regards perspectives as tools, rather than world-encompassing articles of faith. It doesn’t dictate, it listens. It focuses on deficiencies, without making lazy assumptions as to whether those deficiencies are random accidents, just “one of those things,” or some one person’s fault, like the manager. It focuses entirely on removing any and all obstacles to winning, and doesn’t take “shit happens” for an answer, unless the answer truly is “shit happens.”