Jay Jaffe at Baseball Prospectus, in his recent chat, showed what happens to analysts who look at numbers but not the reasons for them. Two excerpts:
Joe (35th and Lowe): White Sox: Legitimate 100-105 win team, or have they just gotten pretty lucky over the past 14 months? Jay Jaffe: Once again, the WHite Sox are playing above their heads; they're +4.7 in the third-order Wins department according to our Adjusted standings. They've been more than a bit lucky over the course of that span, sure, but I do see them as about a 90-95 win team, particularly so long as Contreras and Thome remains healthy.
As I understand third-order wins, they are extrapolating from basic statistics and adjusting for quality of opponents. In other words, if all else is equal, what do the hits, walks, homers, etc., project as for wins. I suppose they have studies showing that their methodology is a sound prediction method, but as far as the White Sox are concerned, they've been basically, insistently wrong for over 200 games now. Further, having followed all of those games, I just can't see where the "luck events" occurred. Jaffe and crew should consider the possibility that their methodology is all approximate, that the real game of baseball is not played by APBA cards, and that when their prediction models can't account for performance, there is a 50% chance that the problem lies in the prediction model.
Nick from WH (SF): What do you make of Jhonny Peralta's slow start? Is he still the real deal? Jay Jaffe: I'm a huge Peralta fan; in fact I think I picked him as my AL MVP in the staff preseason picks. Which is probably the problem right there.
Where to start. Peralta's 20 homers last season should have come with an asterisk: (* Almost all hit off poor pitching). A look at Peralta's pitchers-faced profile showed that a disproportionate number of Misplaced H's home runs came off mop-up pitchers in massacres. Now, there's nothing wrong with this, but expecting that sort of performance to translate to the 3-hole in competitive games was naive, and showed that "one of the five best GMs in the game", whose superior team is now struggling with .500, didn't have deep insight into the true nature of his shortstop.