Evaluating the Moneyball draft eleven years later.Part 1: Making the majors or not

Michael Lewis' book Moneyball concentrated on the Oakland A's in 2002, more on the front office than on the players on the field. The idea was that general manager Billy Beane and his employees were looking at new and better ways to develop talent and use it, giving them a chance to be competitive with teams whose payrolls were several times higher than what the frugal A's ownership was willing to spend.

Beane and his team used sabremetrics, a word coined from the acronym SABR, the Society of American Baseball Research. The idea was that he would be able to draft players under-appreciated by other clubs and build a nucleus of young talent, though the best would be lost to free agency within a few years.

Beane's method really wasn't that scientific. What Billy Beane wanted to avoid was drafting another Billy Beane. He was much sought after out of high school. He debated whether he would go to college or go straight into baseball out of high school. He was drafted out of high school and he made the major leagues eventually, but he wasn't the All-Star the scouts had hoped he would be.

Beane the general manager drafted no high school players in 2002, worried there was a high probability he might find too many kids like himself who crumbled under the pressure of professional baseball.

Using the 2002 draft as our data set, let's ask the question: Is there a significant difference between the success rates of high school and college draftees?

The null hypothesis: There is no significant difference.
Data set #1: The first 50 position players drafted
Data set #2: The first 50 pitchers drafted
How we split the sets: A player was either drafted from high school or college and the player either made the major league roster or did not.
We will perform a chi-square test to see if the differences we see are significant.

Problem with this test: We are lumping together some players with very good careers so far with some guys who just barely had a cup of coffee in The Show. That problem will be addressed in the test used tomorrow.

Position players
============
High school draft: 13 made the majors, 13 did not
College draft: 13 made the majors, 11 did not

Test statistic: chi square = 0.087, well below even the 90% confidence threshold of 2.706


Pitchers
======
High school draft: 13 made the majors, 10 did not
College draft: 12 made the majors, 15 did not

Test statistic: chi square = 0.725, well below even the 90% confidence threshold of 2.706

These numbers just count whether players will make it to the majors or not, and as we can see, out of the first hundred or so players chosen, about half will see major league experience and high school draftees are not significantly different from college draftees. Another question is how good are those major leaguers when we compare the high schoolers to the collegiates?  Tomorrow, we will use a different statistical test on only one stat per player, not a completely fair test, but it does give an approximate idea of the players' worth to their squads.