How to Experiment

This chapter aims to put the remainder of the book, that is, Parts II and III, into context. For this purpose, it focuses on describing the steps to be taken to run an experiment, the most important of which will be described in the other parts of the book. Beforehand, section 3.2 examines what sort of relationships among variables can be outputted by an experiment. Having described these relationships, the process of stepwise refinement involved in any experimentation process is described in section 3.3. Each cycle of this process involves running a given experiment, the process to be followed is described in section 3.4. We will see that this process is composed of the phases of goal definition, design, execution and analysis. All these phases are essential for the success of the experiment. However, experimental design and analysis call for special attention. Therefore, Parts II and III of the book focus on these two phases, respectively. Finally, section 3.5 describes what can be deduced at the end of these stages and what role statistics plays in determining the above conclusion.