Basic Terminology (Sections 1 and 3.1)

(Research) Hypothesis
    Ex: "Visualization A and B are equally good, i.e.
           there will be no difference between users who use
           visualization A and those who use B"
          "Users who use visualization A will perform easy tasks
           faster than those who use visualization B"

Experiment

Treatment variables (aka independent variables, factors)
             controlled (systematically varied) by experimentors
             Ex: kind of visualization interface, difficulty of task

Dependent variables     Ex: task performance time, user satisfaction,
                                              error rate
Nuisance variables (aka confounding factors)
                  Ex: age, fatigue, motivation, gender, time of the day
                    May be hidden: Ex: • Visualization interface A has design flaws
                                              (that are independent of the visualization principle)
                                            • Interface A is always tried in the mornings,
                                               Interface B in the evenings

                          * Balancing these variables across treatments or
                          * keeping them constant across treatments or
                          * (for subject factors): random assignment to treatments or
                         *  factoring them out during the statistical analysis

Subject sampling

Assignment to treatment (block randomization!)