Tuesday, June 13, 2006
Ability and Performance
Today we started talking about learning from data. Here are the main things we talked about:
- There is a distinction between a player's ability and his performance. An ability is an intrinsic quality of a player, say his batting talent, that we really don't know exactly. We do observe a player's performance, say his batting average for a particular season.
- The objective of Statistics is to learn about a player's ability on the basis of his performance.
- Suppose we know a player's true ability to get on base. We describe this ability by a number p that represents his probability of on-base. If we know the value of p, say p = .4, we can predict how many times he will get on base in 20 PAs. We simulated this process in class by the use of 10-sided dice.
- Above we knew a person's ability p and we predicted his on-base percentage in 20 PAs. The statistical inference problem goes in reverse. Suppose we observe a player get on-base 8 times in 20 PAs -- what does that tell us about his true on-base probability p?
- In Fathom we did a simulation experiment in two steps: first, we simulated a player's true batting average p, and then we simulated the number of times on base given that value of p. By classifying all simulations by p and his observed AVG, we saw what we could learn about p based on his AVG in 20 AB.