Randomized Experiments


Experimental Design


During the next section of the course, we will be discussing experimental design.  Specifically, we will be discussing different ways that one can set up a psychological experiment, along with the pros and cons of each.  Our first topic is one of the simplest and most common types of experiment.


Randomized Two-Group Experiment


A randomized two-group experiment has a single independent variable with two levels.  These are the “two groups”—also called the two conditions.  For example, an experiment on the effect of noise on concentration might have a quiet condition and a noisy condition.  Note that even though the two conditions are sometimes referred to as “groups,” this does not mean that all the subjects in a condition must be tested at the same time in the same place.  Even if you tested one subject per day in the quiet condition for each of 20 days, we would still say that these 20 subjects constituted the “quiet group.”


Of course, there is a sample of subjects that will be tested.  Furthermore, half of this sample is assigned to each level of the independent variable.  In the noise-concentration experiment, for example, half the subjects would be assigned to the quiet condition and half to the noisy condition.  Note that in this design, subjects are tested in only one condition, which makes it a between-subjects design.  (The alternative to a between-subjects design is a within-subjects design, in which each subject is tested in each condition.  We will learn about within-subjects designs in a few days.)


A crucial feature of the randomized design is that subjects are assigned randomly to the two conditions.  This is referred to as the random assignment of subjects to conditions.  For the assignment of subjects to conditions to be random, two conditions must be met.


1) Each subject must have an equal chance of ending up in one condition or the other.  For example, for each subject, you might flip a coin.  If the coin lands heads, you assign the subject to the quiet condition; if it lands tails, you assign the subject to the noisy condition.  This would give each subject the same chance of ending up in, say, the quiet condition as ending up in the noisy condition. 


2) Each subject’s chances of ending up in one condition or the other must be independent of the chances of other subjects.  For example, it is not random assignment if you split a group into those on the left half of the room and those on the right, and then you flip a coin to determine that those on the left will be in Condition A and those on the right in Condition B.  This is because each subject’s chances of being in one condition or the other are linked to those of the other subjects.  A subject on the left side of the room is guaranteed to be assigned to the same condition as the other subjects on the left side of the room (and the opposite condition as the subjects on the right side of the room).  Their chances are not independent.




Although it may not be obvious at first, random assignment is a way of controlling for extraneous variables.  This is because it results in groups that are fairly similar overall.  For example, imagine that you randomly assign 50 subjects to each of the quiet and noisy conditions.  You can be pretty sure that there are roughly the same number of men and women in the two conditions, that their average IQ is roughly the same, that on average their hearing sensitivity is roughly the same, and so on.  For this reason, any differences between the two groups on the dependent variable can be attributed to the independent variable, and not to some extraneous variable.


This is also why not using random assignment is bad.  It can result in confounding variables.  Imagine, for example, that we assign subjects sitting on the left half of the room to one condition and subjects sitting on the right half of the room to the other condition.  We run the risk that there is some reliable difference between these two groups that might affect our results.  For example, what if the door is on the left side of the room, so that the left half contains a disproportionate number of subjects who are interested in finishing their task quickly and the leaving?  Or what if, because the door is on the left, the most punctual students tended to sit on that side … with the lazy slackers on the right.  In fact, there might be all sorts of differences between left-side-sitters and right-side-sitters that we could not even think of if we tried.  We sidestep this problem completely, however, by using random assignment. 


Does random assignment absolutely guarantee that the two groups will be similar in terms of all important extraneous variables?  No.  For example, it would still be possible for most of the more intelligent subjects in the sample might end up in the quiet condition, while most of the less intelligent ones ended up in the noisy condition.  But with a large enough sample size, this is very unlikely.  Also, if this does happen, it is a fluke and is not the fault of the experimenter.


Some Other Considerations


Equal group sizes.  Most experimenters consider it desirable to have equal group sizes.  But a random assignment procedure like flipping a coin for each subject is unlikely to result in exactly the same number of subjects in each condition.  There are ways of dealing with this, though.  For example, you can flip a coin for the first subject to assign him or her to a condition, and then you can assign the next subject to the other condition, and you can keep going in this way.  Note that technically this violates the “independent chances” criterion, but it does so in a way that is unlikely to cause problems.


When to do random assignment.  It might be inconvenient to be flipping coins or engaging in some other random assignment procedure at the time of the experiment.  For this reason, you might set up a randomization scheme ahead of time.  For example, before you actually do your experiment, you might flip a coin to determine that the first subject will be in the quiet condition, so the second will be in the noisy condition.  Then you flip the coin again to determine that the third subject will be in the noisy condition, so the fourth will be in the quiet condition.  This way you can have a complete list before the experiment begins, so that as you encounter each new subject, you can simply assign them to the next condition on the list.


How to do it.  Flipping a coin is only one way to randomly assign people to groups, which works well when there are only two groups.  But you can use any method that accomplishes the same thing.  You can draw slips of paper out of a hat, you can consult a list of random numbers or use a random number generator (see http://www.random.org/), etc.  For example, for an experiment with four groups, you could generate a random number between 0 and 1 for each subject.  If it is less than .25, they go into Condition A; if it is from .25 to .50, they go into Condition B; and so on.  


Testing groups.  Sometimes it is desirable to test subjects in small groups for efficiency.  In many cases, everyone in the group must be in the same condition (because you have to give them all the same instructions, present them with the same stimuli, etc.).  In this case, you can no longer randomly assign subjects to conditions.  You have to randomly assign groups to conditions.  For example, you might assign your first group of ten subjects to the quiet condition, your second group of ten subjects to the noisy condition, etc.  This generally works OK, but be careful.  If you only have two groups and you end up testing the quiet group in the morning and the noisy group in the afternoon, you have a serious confounding variable.  It would be better to use more groups and smaller groups.  The closer you come to true random assignment—with each individual subject randomly assigned to a condition—the better off you are.