Section 2.6: Designing the study
From Research Methods in Psychology
Contents |
[edit] WHAT ARE YOU STUDYING?
THIS SECTION LOOKS AT SOME OF THE ISSUES IN DESIGNING YOUR STUDY. THERE ARE A NUMBER OF CHOICES TO BE MADE, AND ANY DECISION THAT YOU MAKE WILL HAVE ADVANTAGES AND DISADVANTAGES, WHICH YOU NEED TO BE AWARE OF. FOR CERTAIN EXPERIMENTAL DESIGNS, YOU NEED TO TAKE MANY MORE PRECAUTIONS, WHICH WE WILL CONSIDER HERE.
[edit] Why do you need to know how to design an experiment?
To test a hypothesis, we need to design and carry out a study. When you plan a study, you will need to make a large number of decisions. The decisions that need to be made do not necessarily have any specific right or wrong answers - there will be pros and cons to every answer. (Of course, some answers will be better than others, and some will be wrong, but none will be exactly correct.)
You are likely to be assessed on your understanding of experimental design, and your understanding of what the advantages and disadvantages of different designs might be.
- You will be asked to design studies. In designing a study, you need to justify why you made the decisions that you did, and why you rejected other options.
- You might be given descriptions of studies, and asked to evaluate the choices they made. You also might be asked to provide alternative designs, and evaluate those choices.
Once that a psychologist has proposed a hypothesis the next stage is to design an experiment or study that will test that hypothesis. We will look at two different issues in this section: the experimental setting (the field, or the laboratory) and experimental design.
[edit] Field and Laboratory Studies
When designing a study a researcher has to decide whether to carry out the research as a laboratory study or as a field study. The choice of one technique over another often sacrifices the ability to collect clear data on some aspect of the problem. The choice of technique therefore, will usually be a compromise.
| Crucial Concept: A laboratory study takes place in an environment where the researcher has a great deal of control over the surroundings - in a psychology laboratory. |
| Crucial Concept: A field study takes place in an environment where the researcher does not have a great deal of control over the surroundings - outside a psychology laboratory, in the ‘real world’.. |
In the laboratory environment an experimenter is able eliminate or control for possible extraneous variables, so that the experimenter can be fairly confident that any changes that are measured in the dependent variable or caused by manipulations of the independent variable(s). For example, if we wanted to test participants’ ability to solve two types of reasoning problem (one easy and one difficult) we might choose an independent groups design.
In some instances, however, a laboratory environment may be rather artificial. For example, a developmental psychologist might want to observe the way in which siblings play with a particular toy. Placing children in a laboratory situation would allow clear observations to be made without the children knowing that they were being observed. However, taking brothers and sisters out of their home environment and putting them into a laboratory situation might inhibit their natural play behaviour and/or promote less natural play behaviour. In this instance, the experimenter couldn’t be confident that the behaviour observed was not in some way attributable to the laboratory environment.
This observational study, therefore, might best be carried out in the field. This could involve the experimenter observing the children as they play at home. The home environment might promote more natural play behaviour than in the laboratory, but there is a risk of the children being inhibited by the presence of the experimenter. Here we can see how the choice of whether to carry out a laboratory study or field study may sometimes be a compromise.
[edit] Experimental Designs
If you have decided that an experiment is the best approach to testing your hypothesis, then you need to design the experiment. Sometimes the choice of experimental design will be a very straightforward choice, on other occasions it will be more difficult to decide. There are two major types of design that we will consider here: repeated measures and independent groups. There are other designs, which are much less commonly used and we will consider them at the end of this section.
[edit] Independent Groups Design
| Crucial Concept: An independent groups design (also know as a between-participants design) involves each participant being tested under only one level of the independent variable. This means that each condition of the experiment includes a different group of participants. |
It is very important that participants are randomly assigned to the different conditions, because by random assignment we can to each condition of the independent variable. If they are not randomly assigned, we cannot be sure that any differences which arise between the groups are not due to differences between the groups, and are due to the independent variable.
[edit] Repeated Measures Design
| Crucial Concept: A repeated measures design (also know as a within-participants design) involves each participant being tested under all levels of the independent variable. This means that each condition of the experiment includes the same group of participants. |
With the repeated measures design each participant is tested under all levels of the independent variable(s). This means that each person is tested on more than one occasion. Repeated measures designs are sometimes known as within-subjects designs, because we are making a comparison within one group of people.
The major advantage of the repeated measures design is that each subject effectively acts as their own control - this means that fewer participants are required, to achieve the same degree of statistical power (see Chapter 4 for a discussion of statistical power).
[edit] Problems with Repeated Measures Designs
Some problems can arise when using a repeated measures design. Keren (1993) identified them as: Practice Effects: A practice effect occurs when a participant has practiced a task, and will be better at it the second time they do it. If we ask people to solve some anagrams as quickly as they can, the second time they do it, they will be more practiced at solving anagrams, and will do it more quickly.
Sensitisation: If you are assessing factors that might change people’s attitudes or beliefs, you might ask people the same questions on two occasions. However, participants may keep their answers similar - they might want to seem consistent. If I said that people who are caught in possession of cannabis should be automatically fined £100, whatever you say to me, I will remember that I said that, and will not change my statement, even if you do succeed in changing my beliefs.
Carryover Effects: A carryover effect occurs when the fact that an individual has already done part of the experiment will affect how they perform in the next part. The most important carryover effect is fatigue. By the second or third time that a person is running through an experimental procedure, they are likely to be getting bored or tired. Some treatments (for example drugs) may remain in the system of the participant, or have a long-term effect. Carry-over effects will also occur if the experimental procedure involves a deception that is revealed to the participant as part of the procedure. Some studies on memory ask a participant to read a list of words, and answer questions on those words. When they have done this, the participant is asked to recall the as many of the words as possible. A repeated measures design would be no use at all here, because the second time the participant were asked to undertake the task, they would expect their memory to be tested and would therefore make a much greater attempt to recall the words.
[edit] Solutions to the Problems
Counterbalancing and Randomisation
Counterbalancing and randomisation are two precautions that can be taken against the problems identified above. Crucial concept: Counterbalancing involves equal numbers of participants doing the conditions in each of the possible orders of the conditions.
This is very straightforward when we have just two conditions. Half of the participants do one condition first, and the other half do the other condition first. You would divide your participants (randomly, of course) into two groups, A and B. Group A would do condition 1 first, and Group B would do condition 2 first.
With more than two conditions, things become a little more complicated. We will look at the example with three conditions.
You split the sample (again randomly) into three groups, Group A, Group B and Group C. Group A carry out the conditions in the order 1, 2, 3, Group B carry out the tasks in the order 2, 3, 1, and Group C carry out the conditions in the order 3, 1, 2.
Before reading on can you think of any problems arising from this method of counterbalancing?
The problem is that Condition 2 always follows Condition 1, and Condition 3 always follows Condition 2. It might be the case that there is some ‘carry-over’ from one condition to the next, and because the conditions always follow the same order, may be a systematic bias - affecting one condition more than it affects the other conditions. It is possible to avoid these problems with more complex counterbalancing, known as a Latin Square design. An alternative method is randomisation.
| Crucial Concept: When using randomisation, the conditions for each participant are placed into a random order. |
Randomisation ensures that any systematic effects of order should be minimised, if not eliminated. In a study of memory, you might be interested in presenting participants with two types of words, to see which type is more likely to be remembered. If you were counterbalancing, you would present first one list, and then the other (varying the order). For this type of study, a better approach may be randomisation, where you mix the words together and present them random in a random order, which would differ for each participant. Writing each word on a card, and then shuffling the cards is the easiest way to do this, although it can also be done using a computer based presentation.
| Crucial Tip:If you have carried out a study using a repeated measures design, it is vital that you use counterbalancing. When you are writing a practical report, if you used a repeated measures design, you need to describe how you counterbalanced. |
Pre-Practice
Figure 3: A Typical Learning Curve
The second precaution which can be taken when using a repeated measures design is to pre-practise. Most skills go through a learning curve where skill develops rapidly at first and then more slowly, as shown in Figure 3. If each participant practises the task before the experiment, the effects of practice (and of different practice between conditions) will be minimised.
[edit] Section Summary
In this section, we have considered some of the issues that arise when designing experiments. First, we considered the issue of experimental setting: experiments can be set in the laboratory, for greater control, or in the field, for greater realism. The next issue that we considered was the issue of using an independent groups design, or a repeated measures design. A repeated measures design requires fewer participants than an independent groups design, but raises problems with practice effects, sensitisation and carryover effects. Sometimes precautions such as counterbalancing, randomisation and pre-practice can eliminate these problems.
[edit] Further Reading
Many texts that discuss experimental design also discuss the associated statistics (there is nothing wrong with that - the two cannot really be separated, but you as the reader are only on Chapter 2). If you don’t mind this, then a nice little book on design is “Relating statistics and experimental design: an introduction”, by Levin (published by Sage). Most introductory books on research methods and statistics look at experimental design. Your lecturers may already have recommended a book to you, but if they have not, you could have a look at “Research methods in psychology” by Heiman (published by Houghton Miffin). A very clearly written book is “Methods of Research in Social Psychology”, by Eliot Aronson, et al., (published by McGraw Hill) - don’t let the social psychology in the title put you off (if it were about to), what they say applies to most experimental research in psychology. If you are interested in going into much more detail of research in field settings you could look at “Quasi-experimentation” by Cook and Campbell, (published by Houghton Miffin), or the new edition, by Shadish, Cook and Campbell.
