Multiple comparisons

With each pairwise comparison there is probability (α) to take the wrong decision, if all means are equal (an error of the first kind). The total probability of being wrong at least once, in that case, will be larger than α.

With a multiple comparison  procedure it is attempted to control this so-called experiment-wise error below α .

There are numerous procedures, e.g. the Bonferoni procedure, Tukey’s procedure, Dunnett’s procedure, … etc. We discuss here only Tukey’s procedure.

When a multiple comparison procedure is needed, for all pairwise comparisons, Tukey’s  procedure is a sensible choice.

Tukey’s procedure uses the so-called studentized range distribution (table in books).

Two treatments differ significantly, when the difference between their sample means exceeds the yardstick W; |y ?_1-y ?_2 |>W, and W=q√(s^2/n)

q depends on the number of treatments (t), the residual degrees of freedom (N-t), and the desiried experiment wise error rate (α). The value of q (the studentized range distribution) can be found in tables (and software).