Section 2.2: The structure of experiments
From Research Methods in Psychology
[edit] What are you studying?
THIS SECTION BRIEFLY INTRODUCES THE IDEA OF AN EXPERIMENT, AND EXPLAINS WHAT MAKES EXPERIMENTS DIFFERENT FROM OTHER RESEARCH TECHNIQUES. WE INTRODUCE SOME OF THE IMPORTANT ASPECTS OF EXPERIMENTS.
[edit] What is an experiment?
In layperson’s terms, an experiment is an attempt to find something out, or to investigate something. In psychology, the word experiment has a more specific meaning, and when you are doing psychology, you should make sure that you use the word in its correct, technical sense.
In psychology, a hypothesis is a statement usually in the form ‘X affects Y’.
| Concept: An experiment is carried out to test a hypothesis. |
Does X indeed affect Y? Suppose our hypothesis is ‘Noise affects ability to carry out a mathematical task.’ How do we find out? We ask people to do mathematical tasks. We inflict noise on them while they are doing the tasks. We get the test results and analyse under what noise conditions the results were good and under what conditions the results were not so good.
| Concept: The independent variable (often abbreviated to IV) is the variable that we manipulate. |
In the example above, the independent variable is the presence or absence of noise.
| Concept: The dependent variable (often abbreviated to DV) is the variable that we measure. |
In the example above, the dependent variable is the success our subjects have in doing the mathematical task - their test scores. Our alternative (experimental) hypothesis states that by changing the IV (noise), we will have an effect on the DV (results of the maths tests). The structure of a simple experiment is shown in
Figure 1.
Figure 1: The relationship between the independent variable and the dependent variable.
| Concept Tip
It is in experiments, and only in experiments, that we manipulate an independent variable and measure a dependent variable. If you (or someone else) did not manipulate an independent variable, and measure a dependent variable, they did not do an experiment. |
However, the independent variable (in our experiment, the noise) is not the only thing that influences the dependent variable (in our experiments, the maths test results).
|
Concept: Extraneous variables are additional variables that influence the dependent variable. |
There are two kinds of extraneous variables, and they each have a different sort of influence on the results. Extraneous variables can have random effects and systematic effects
| Concept: Random effects have an effect on all participants, but not in a predictable way. |
If there is a random effect in an experiment, it means that some people’s scores on the DV will be lowered, and some people’s scores on the DV are raised, but the average effect is zero – the effects cancel each other out. The most common random effect that we have to deal with is individual differences - everyone has different abilities at maths. Random extraneous variables are troublesome, but we can live with them if we can minimise their effects.
| Concept: Systematic effects, (sometimes known as biases) have an effect on some participants more than others, and will bias the results of an experiment. |
A variable that causes a systematic effect is called a confounding variable (or sometimes just a confound). A confound, bias or systematic effect is not randomly distributed; rather it affects some group(s) more than other group(s). Because systematic effects influence one group more than they influence others, it will disguise the effect of the independent variable. In the example above, if we allowed the ‘silent’ group more time to do the maths test than the ‘noise’ group, we could not be sure than any differences between the groups were due to the independent variable - they might be due to a systematic extraneous variable.
Let’s summarise what we have been saying, and examine the experiment that we have been describing in a little more detail. Here is the method we used in the experiment to test the hypothesis that noise influences maths test results. Twenty participants were randomly divided into two groups of ten, and each participant was given ten maths problems to solve in ten minutes. The first group were asked to do the problems in quiet conditions, the second were placed in an identical room, and asked to do the problems whilst four radios were playing at the same time, each one on a different station.
Of course, because of random extraneous variables, we expect people to get a range of different scores, but we expect that on average, the quiet group will get higher scores than the noise group. (If there were no random extraneous variables, the quiet group would all get the same score, which would be higher than the score for the noise group.)
Much more serious than the effect produced by the random extraneous variables discussed above is the bias effects resulting from a systematic extraneous variable (or confound). It can be hard work to randomly assign participants to conditions, so instead of using random assignment, we assign people to one group or the other in the order that they turn up for the experiment. The first ten people will be assigned to the Quiet condition, the second ten people will be assigned to the Noise condition. This method is, after all, much easier than assigning people randomly, and surely it will make no difference?
First, yes this method is much easier. Second, this lazy method might well make a difference to our results. Why should it make a difference? To see why, we have to consider why some of the people turned up later than other people. Are there really no differences between people who arrive on time, and the people who arrive late? It might be that the people who arrive late are generally less conscientious about many things. The people who arrived late maybe were late in getting up, because they went out last night, and so they are tired, possibly hung over, they may not have had breakfast or brushed their teeth, and will generally be feeling worse. It might be the people who arrived late had been at a special class for high achievers in mathematics that was in the other building; or it could be that they had been attending an additional lecture, designed for students with hearing difficulties. The trouble is, we do not know the reason for their lateness, and if we don’t know the reason, we cannot rely on the late people being the same, for our purposes, as the early people.
Simpson’s paradox (Simpson, 1951) is nice example, which shows the importance of random assignment of participants to conditions, and we explore this in the box. Box “Simpson’s Paradox” and random assignment
| (This is a box, which contains some additional information. It won't hurt to skip it).
Simpson’s paradox is an interesting puzzle in methodology, which shows how the importance of random assignment of participants to conditions. The hypothetical experiment below shows Simpson’s paradox in action. This example, from Pearl (2000) gives the results of a simple experiment to examine whether taking a headache pill reduces a headache are shown in the table below. It shows that one group of 40 participants either took a pill for their headache, and of those 40 people, 50% have been cured. A second group of 40 participants took a pill for their headache, and we find that only 40% of these people have been cured. It seems clear from this that we would suggest that taking a pill is likely to cure your headache. All Participants
But the researchers who carried out the experiment did not randomly assign people to the two conditions. When they did the analysis with males only, they found the results shown in the table below. So for the males, the drug did not seem to work - 70% of those who did not take the pill were cured, and only 60% of those who did take the pill were cured. Males Only
“OK,” said the researchers, “it doesn’t work for men, it just works for women.” So, as a double check, they did the analysis on the women only. The results are shown in the table below, and as you can see, of the women who took the pill, 20% were cured, whereas of the women who didn’t take the pill 30% were cured. Females Only
This seems to be impossible! The works for ‘people,’ yet it doesn’t work for men, and nor does it work for women. So, what is going on? The answer is that the researchers did not randomly assign participants to conditions. A higher proportion of men decided to take the pill (30 out of 40 men decided to take the pill), only a small proportion of women decided to take the pill (10 out of 40 women decided to take the pill). And men were more likely to get better, whether they took a pill or not. |
A summary of the experimental design process is shown in Figure 2. The independent variable (which we manipulate) affects the dependent variable (which we measure). Extraneous variables also affect the dependent variable. We should attempt to remove all confounding variables (biases), and to minimise the effects of random variables. In the following sections, we will have a closer look at each of the types of variables (independent, dependent and extraneous), and consider some of the choices that you have to make when you are designing and carrying out experiments.
Figure 2: The relationship between the IV and the DV, including extraneous variables.
[edit] Section Summary
This section has introduced the important features of experiments that we will look at in more detail in the rest of the chapter. Experiments feature an independent variable (IV) which is manipulated by the experimenter, a dependent variable (DV) which is measured by the experimenter. Extraneous variables also affect the DV, and we aim to remove, or minimise, their effect. One of the most important aspects of experimental design is random assignment of different participants to different conditions – if you do not randomly assign participants to conditions, you cannot be sure that and differences in the conditions are due to the independent variable.
