Improving statistical inferences 1_2
What is a p-value?
1. Why is the p-value so successful in science? In some sense it offers a first line of defense against being fooled by randomness, separating signal from noise.
2. P-value tell you how surprising the data is, assuming there is no effect.
3. e.g. The difference is never exactly zero. A difference of e.g., 0.11 means:
A) Probably just random noise.
B) Probably a real difference.
From the M, SD, and N, we calculate a test-statistic, and compare it against a distribution. (examine precognition use a paranormal distribution).
(A p-value is the probability of getting the observed or more extreme data, assuming the null hypothesis is true).
A p-value is the probability of data not the probability of a theory.
4. P(D * | H) != P(H | D) (if you want to know the probability that the theory is true, you need to use Bayesian statistics.
5. When there is a true effect, the p-value distribution depends on the statistical power.
6. When there is no effect, p-value are uniformly distributed