Fielding Statistics are Totally Useless
A while back I was harping on about the extreme variability of fielding statistics, which pretty much came down to a big difference in the data collected by Baseball Info Solutions and STATS. The example I used was Troy Glaus, who 0ne company says is one of the best defenders in the league and the other claims is one of the worst. Well, I haven’t even looked at anyone other than the Jays, and here’s another example…
Despite winning a Gold Glove for the last three seasons, Vernon Wells never gets great marks from the defensive metrics (especially in 2005, when his fielding percentage was 1.000 but no matter who you ask his range wasn’t great). This season the numbers from BIS rank him as the worst centre fielder in the American League. He has a terrible Zone Rating, and the fewest balls caught Out of Zone (OOZ) than anyone in the league, most by a large margin.
STATS, on the other hand, says he was one of the best. Breaking down the numbers, STATS says Vernon got to 317/348 chances, while BIS gives him 289/329 on Balls in Zone and 32 OOZ, for a total of 321/361. Wells had 321 putouts this season, so that makes sense for the BIS numbers- I have no idea how he could get more putouts than chances under the STATS system.
So at least in this case, it looks like the CF Zones are smaller than the BIS ones- the opposite as it was for Glaus at third. Again, we could do a lot of work looking at retrosheet for the positions to try and figure out why one system has him making more plays and even compare the results for every player to figure out how much of a difference there are between the zone definitions, but I’m more interested in baseball than trying to statistically analyze what some company won’t tell me about why its data is right or wrong.
Again, I ask the question: even though the complicated metrics dissect and manipulate the raw data taking into account every possible factor (how hard balls are hit, from which side of the plate, etc.) how can we consider the rankings they come up with particularly useful or accurate when the two main sources of raw data keep their methods under wraps and reasonably often differ 100% on their evaluation of a player?