Decision rules and errors (Chapters 5 and 8.1)

Critical value Fa for significance level a: that F value where the cumulative probability distribution reaches 1-a (in other words, the probability that F takes on a value larger than the critical value is a)

Typical significance levels: 0.05 (=5%, "significant")
                                          0.01 (=1%, "very significant")

Null Hypothesis H0: no difference exists between groups
Alternative Hypothesis H1: there is a difference between groups

Let Fobserved be the observed F value.
If Fa > critical value, reject H0 and accept H1, otherwise retain H0 and reject H1.

Type I (a) error: reject H0 even though it is true (wrongly assume treatment effect)

Type II (b) error: accept H0 even though it is false (fail to recognize treatment effect)

Power of an experiment: 1-b: ability to recognize treatment effects

How to reduce a errors:
- Increase the significance level (this increases the type 2 error though)

How to reduce b errors:
- Increase the sample size
- Increase the size of the treatment effect
- Decrease the amount of experimental error
- Use a more sensitive experimental design