The Eye Test

My main computer is still down – I bought a Macbook pro in 2011, and it’s crashed big time. Not good. As a result, computer access remains limited and problematic.

I am told it will be fixed by Monday. As my daughter notes, there are *many* worse things that can happen to a person.

On Wednesday, Mike Greenberg had a complaint about a comment related to the newly constituted committee that will pick the four participants in next year’s first ever major college football playoff. There’s been lots of commentary in particular about the inclusion of former Secretary of State Condoleezza Rice’s on the committee. I am going to leave that to the side. What had Greenie exercised was the statement by former QB and now college football analyst Danny Kannell that anyone who was going to choose the playoff teams would need to watch all the games, that only the “eye test” could really tell you who earned a spot in the pool.

Greenberg argued, rightly, that if you are going to have a selection process, you need objective criteria by which to determine what teams are better. Yes, deciding what criteria to use involves necessarily arbitrary decisions. But it is possible to make some determinations about which criteria are more important than others, rather than leaving it to each member of the committee to decide for themselves what teams are better than others based on watching them.

As I’ve said before, the value of using statistics to evaluate performance, among other things, is that, in fact, our eyes (and hearts and minds) deceive us all the time. We develop beliefs, for example, about how players perform in crunch time situations based on how they carry themselves, the reputations they develop in such circumstances and so on. On this basis, Kobe Bryant is considered a legendarily clutch performer, even though the statistical record suggests he shoots quite poorly in clutch situations.

Statistics are, themselves, constantly misused. But the notion that our best guide to performance is what we see is deeply flawed. I have complained to a friend of mine for years about sports commentator who say you can’t know whether a baseball players is worthy of the MVP award unless you see them play every day. Anyone who makes such a statement should, in fact, be immediately disqualified from ever voting on the MVP, since it would not be possible to watch every potential MVP candidate play every game (and no one does). But beyond that basic logical fallacy, the larger issue is that we don’t have infallible judgments. We filter, fixate and fuss over this or that outcome, mannerism, style of play and so on. We are distractible in countless ways. As a result, it doesn’t make sense to make the so-called eye test the primary basis for judging performance.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s