How much do PEDs help performance?

David Schoenfield, long-time ESPN editor and baseball writer had a worthwhile column this week about the above question. Schoenfield begins:

All the anger, the venom, the indignation directed towards Alex Rodriguez comes from an essentially unproven pretense: That performance-enhancing drugs actually enhance performance. In baseball, that basically means one thing: home runs. Nobody seems to care all that much whether PEDs help players run faster or field better or throw a tighter curveball. It’s all about the home runs.

As Schoenfield goes on to say, the evidence that steroids in particular caused a spike in home runs seems irrefutable. Baseball experienced an enormous spike in offense beginning in 1993 that lasted for more than a decade. Most baseball fans know how much the numbers from the 1990s and early aughts stand out. Before 1998, two players in baseball history slugged 60 homers or more in a season: Babe Ruth hit 60 in 1927 and Roger Maris hit 61 in 1961. Then between 1998 and 2001, the sixty homer plateau was scaled six times, with two players smashing the there-to-fore cartoonish 70 homer barrier during that stretch (when I was a 10 year old fantasizing about being an all-time great, I never allowed myself to hit more than 64 homers in a season. Sammy Sosa reached that figure twice). As Schoenfield says, in 1996, 16 players hit 40 homers. In 2011, two managed the feat.

In sum, when the drugs flowed freely, prior to testing in the late 1990s, balls were flying out of the park. When baseball started cracking down, home run totals followed.

But according to the respected baseball analyst Joe Sheehan, the standard story doesn’t stand up to closer scrutiny. As quoted by Schoenfield, Sheehan writes:

The big lie is this: Steroids caused home runs and testing stopped home runs. That didn’t happen. I used to think it was laziness that spread the misinformation, and then I thought it was fear of math, perhaps hatred of certain individuals. I know better now. It’s a lie proffered by people who can see the truth but are so invested in the lie that they would prefer you didn’t.

Sheehan’s major piece of evidence is the percentage of balls put in play that went over the fence. Looking at the period 1993-2013, Sheehan found that when batters actually made contact, there was very little variation from year to year in how often those batted balls resulted in homeruns. Presumably, Sheehan argued, if PEDs were making hitters much stronger, then there ought to a significant spike in batted balls leaving the yard. But this did not happen. For example, in 1998, 3.74% of balls put in play were homeruns. In 2012, the figure was slightly higher: 3.83%.

If you look at the chart, you will see some fluctuation (from 1999-2001, the contacted ball/home run ration was a bit over 4%, more than it was before or has been since). Sheehan says that small spike was related to other factors, including the relatively small strike zone then being called and the effects of the recent double expansion (1993 and 1998) on pitching depth. But you won’t find any dramatic changes over time.

JC Bradbury, the sports economist who wrote the Sabernomics blog for a number of years, did an analysis several years ago of Arod’s performance, estimating the likely impact of steroid use on his home run totals from 2001-2003 (after Arod admitted in early 2009 that he’d used PEDs during those three seasons). Rodriguez led the AL in taters in each of those three seasons, hitting 156 in all during that span. According to Bradbury’s best guess, the drugs were probably worth about two dingers per season.

The claim that anabolic steroids, by beefing up players, made them more likely to hit home runs has never fully explained why widespread use of the drugs by pitchers didn’t seem to help their performance. But it’s also often been claimed that certain PEDs, especially human growth hormone, improved eyesight. This might explain, for example, the extraordinary spike in Barry Bonds’ walk total during his historic run from 2001-2004, allowing him twice to shatter Ted Williams’ single season record for on base percentage. But there is significant dispute about whether HGH improves eye sight and, in general, the scientific community appears to be underwhelmed by the evidence supporting the claimed link.

Sheehan’s evidence isn’t dispositive, as the lawyers like to say. As Schoenfield notes, trying to tease out cause and effect in this case is complicated. But he says, and I agree, you have to, at the very least, reckon with Sheehan’s data. Schoenfield adds:

Baseball is constantly changing. Something in the playing conditions changed rapidly between 1992 and 1994, when offense suddenly skyrocketed. It wasn’t just PEDs, unless you think everyone started using the same offseason. PEDs were just one factor, maybe a small one, along with expansion, livelier baseballs, more strength training, smaller strike zones, bad pitching, maple bats and new, homer-friendly ballparks.

But the conditions continue to change, even in the testing era that began in 2004 (anonymous testing was first conducted in 2003). Pitch F/x data has influenced umpires, players are still big and strong and teams are looking for more big and strong players and more new ballparks have been built — Miller Park (2001), Great American Ballpark (2003), Citizens Bank Park (2004) and Yankee Stadium (2009) are all better home run parks than their predecessors. The weather changes. And the pool of players change, making any study of the issue complex; we don’t have the same pitchers and hitters as 1998 or 2001 or 2004.

One of the biggest changes in baseball in recent years has been the climb in strikeout totals, to all time highs. Players appear to continue to pursue an approach that came into vogue in the 1990s and could be said to be a product of the Moneyball philosophy’s effect on the game: more hitters willing to work deep counts and trade strikeouts for the possibility of going deep.  Perhaps the intersection of this approach with a somewhat expanded strike zone in recent years is a factor in the relative decline in offense despite similar levels of batted balls leaving the park. In any event, what’s refreshing about Sheehan’s approach, as is his wont, is that he actually tries to think systematically about the evidence for or against a particular supposition. Moral outrage, when it comes to steroid/PED use, is easy. Questioning whether it’s as big a deal as people claim is a little harder.

Update: Some comments on FB, including by my old friend Dan Friedman have reminded me of a thought I had when I first read the Schoenfield piece and neglected to include in the original post – that one would expect steroids, HGH and so on, like anything else people put in their bodies, to have different effects on different people. Even if, on average, they don’t do for people what is commonly supposed, they might indeed greatly help some players (and actually hurt others). So, perhaps McGwire, Sosa and Bonds, for example, benefited well beyond the average player. Along these lines, one would want to see the data Sheehan presented, but broken out for individual players.