Super Bowls Georg should not be counted

I'm on a football Discord with a bunch of other football nerds. One of the passions of such people is discussing how football teams should be organized to maximize their odds of winning the Super Bowl. How good does your quarterback need to be? Should you put everything you can into getting one of a handful of transcendent quarterbacks, or is it better for most teams to try to find a "good enough" quarterback and build an excellent team around him? If Kyle Shanahan is such a good coach, why hasn't he won a Super Bowl yet?

One of the problems with these conversations is Tom Brady. He won seven Super Bowls, six with the Patriots and one with the Buccaneers. Tom Brady has been present in 40% of all Super Bowls in the last 25 years. Just behind him is Patrick Mahomes, quarterback for the Kansas City Chiefs, with 20% attendance through the same time period. These are stupendous numbers. Most NFL quarterbacks never attend even one Super Bowl — just over 27% of quarterbacks who played in that period made it even once, and the vast majority of them made it only once.

So a lot of these conversations ground out at, "Well, simply select Tom Brady." (In the sixth round of the draft, no less. Just an absolutely ludicrous outlier career on all levels.) You can extrapolate a little bit further than this on some dimensions. A good quarterback makes a huge difference to your Super Bowl odds. If you're trying to win a lot of Super Bowls you need to get one of these guys. A number one pick isn't the only way to get "one of these guys," it's worth taking dart throws on quarterback lower down the board.

But mostly, he's the huge ridiculous outlier and he warps everything around him. For instance, for a while there was a fashion in the NFL for giving the head coach a lot of control over personnel, the "combined head coach and general manager" model, because Bill Bellichick was both the head coach and general manager and look how many Super Bowls he went to. This hasn't worked out for anyone else who has tried it — if anything, the "general manager rules all and conclusively outranks the head coach" model that the Eagles and the Seahawks use has proved a lot more repeatable. In retrospect, it mostly looked like it worked because of the presence of football singularity Tom Brady.

If you're a quarterback yourself I suspect there's a lot you can learn from Brady about what excellent quarterback play looks like. Probably also true if you're a quarterback coach, or even a passing game co-ordinator or an offensive co-ordinator or a head coach. Likewise there are probably things you can learn from watching Brady tape if you're anywhere involved in the defensive side of the ball. What is it that made this guy so uniquely excellent? Brady is particularly interesting as an object of analysis because he did it for such a long time, and on the Buccaneers as well as the Patriots. With some excellent quarterbacks it's hard to tell how much of their excellence is them and how much is the environment. With Brady it's clear that there's a lot he's doing that's him, that transcends the rest of the situation.

But if you're trying to look at a football team as a spreadsheet, as one spreadsheet in a series of spreadsheets, and you're trying to draw conclusions about what makes football teams good in general, I think you have to throw everything with Brady in it away— or at least put it in folder labelled "Super Bowls Georg." Otherwise all the conclusions you draw about what makes a good team will contain the caveat, "oh and by the way, also have Tom Brady."

One of the things I like about talking about football is you get these very clean examples of problems that are also found in other domains but have maybe a thousand other factors and not nearly as clean of a success criteria. The problem I'm interested in here isn't, "Tom Brady is an outlier and you have to be careful about the conclusions you draw from data sets where he's present." It's the reaction that you get when you say, "So obviously we have to exclude the Brady Super Bowls from our analysis of [some aspect of Super Bowls in general.]" People will generally agree that he's an outlier and then object to removing him because you'd have to remove so much data. 40% of all Super Bowls, remember?

It's a bit like if you were talking about the OG (original Georg), Spiders Georg, and you said, "So we need to remove Spiders Georg to understand how many spiders most people actually eat" and the response was, "Sure I understand he's an outlier but if we remove him then we're removing the vast majority of spiders eaten, and how can we be expected to draw conclusions from a data set that's so small?"

My conclusion here is that if you want to know things, actually know things, you have to get comfortable with the limits of what it's actually possible to know. One of the big obstacles I see to most people's ability to know things at all is that they would rather pretend to know things than to drop Super Bowl Georg from the dataset.