When designing an experiment to test a hypothesis, the aim, so far as possible, is to come up with a scenario where the DV (measured result) can be shown to have been caused by the IV (the supposed cause), rather than any other factor.
Depending on the hypothesis being tested, there are different ways that an experiment may be designed. Three common designs are:
What all these designs have in common is that they make it possible to conduct the experiment more than once, with different values for the IV, in order to see how the DV changes. Each of these designs is subject to problems which might mean that the observed results might be caused by something other than the IV, in which case the result might be confounded.
In an IG design, the experiment is conducted with different values of IV on different sets of participants, where the participants, who form a representative sample of the target population are randomly split into two separate groups.
For example, to test the hypothesis that "watching violent tv programmes causes children to have nightmares", an IG design might involve allowing one set of children to watch 10 hours of violent TV programs and counting the number of nightmares they experience, and then comparing that with the number of nightmares experienced by another set of children who don't watch any violent TV programmes.
The main problem with IG is that it may be difficult to know whether differences observed between two groups are in fact due to different characteristics of the members of those groups, rather than a result of the IV. For example, it could be that the group watching violent TV programmes happens to contain a lot of children of a naturally nervous disposition.
In an RM design, the experiment is conducted more than once on the same set of P's, but with different values for IV each time. For example, an RM design for the "nightmares" hypothesis might be to measure the number of nightmares a set of P's experience after an evening not watching any violent TV programmes, and then the next day let them watch violent TV programmes all evening and seeing how many nightmares follow.
RM might appear to solve the main problem of IG, because the same P's are being tested each time. In fact though, after participating in one part of the experiment, each of the P's is different from the person they were before the experiment started: for example, he now has experience of participating in the experiment, is now a bit older, maybe more tired, etc..
Some of the problems with RM are termed order effects. These occur when the effects of performing the experiment once change the way the participants behave when they perform it the next time. For example, if P's watch violent TV programmes for a day then count their nightmares, then watch no violent TV programmes and count their nightmares, it could be that any nightmares they experience on the second night are due to the violent TV programmes they saw on the first day.
In some cases, it may be possible to design the experiment to reduce order effects: for example, get the P's to watch violent TV on the second iteration of the experiment, rather than the first. Another approach would be to split the group in two, and have different halves of the group performing the two versions of the experiment in different orders, which would hopefully cancel out the order effects. This is called counterbalancing.
An MP design is similar to IG in that there are two groups of P's, but in this case each member of one group is matched to someone in the other group in terms of aspects of their character which could be relevant to the experiment. For example, in the "nightmare" case, a questionnaire might be used to assess each person's inherent "nervousness", so that for each neurotic person in one group, an equally neurotic person could be put in the other group.
The main problem with the MP design is that it is very difficult and time-consuming to match people up in terms of all the characteristics that could be relevant.
In fact, this is such a problem that the MP design is typically only used in cases where the IV is something that the experimenter cannot control - a naturally occurring difference, such as gender. For example, a hypothesis such as "women drink more coffee than men" could only be tested with MP - in this case, you'd have to do the best you could to achieve matched pairs, because you can't test such a hypothesis with IG or RM.
Were it not for order effects, RM would be the "ideal" experimental design, and so where possible, it should be considered first. If it is not possible to design an RM experiment in a way that successfully counterbalances ordering effects, then IG would be the second favourite. MP should typically only be considered if the IV is something that cannot be controlled by the experimenter.
In any experimental design, there needs to be awareness of situational variables - that is, aspects of the experimental setting which may change from one iteration to the next, and that could affect the result of the experiment. These split into two types:
Aspects of the environment over which the experimenter has no control, such as temperature, noise, weather.
Aspects of the experiment which can be controlled, such as the tone of voice which the experimenter uses when delivering instructions.
Since these variables could affect the experiment, every effort should be made to ensure that they do not change from one iteration to the next. For example, if noise is an issue, then conduct the experiment in a sound-proofed room.
Back to class notes index page
Read through the sheet "Methods in Psychological Research"