Carryover Effects and Counterbalancing
The problem with within-subjects designs is that they are subject to carryover effects or order effects. This is when having been tested under one condition affects how participants behave in another condition. There are many different kinds of carryover effects. Here are a few of the most important.
1. Practice Effects – Occur when subjects get better at the task over time because of practice, so that they perform best in the later conditions.
2. Fatigue Effects – Occur when subjects get worse at the task over time because of fatigue. They might even quit trying and just “go through the motions.”
3. Assimilation and Contrast Effects – Occur when a stimulus is perceived as particularly similar to a preceding stimulus (assimilation) or as particularly different from a preceding stimulus (contrast). An example of assimilation is that a poor essay that is presented after a good essay might be perceived as better than it normally would be … because it is associated with the good essay. An example of a contrast effect is that a noisy condition experienced after a quiet condition might be perceived as even noisier than it normally would be. Or a stimulus person who is not smiling presented after a stimulus person who is smiling might be perceived as especially unfriendly.
4. “Catching On” Effects – This is a term I made up to describe situations in which being in more than one condition makes it clear to subjects what the independent variable is, so that they “catch on” to the hypothesis being tested. In many cases, subjects will then behave the way they think you want them too. Some disagreeable subjects might behave just the opposite!
Another kind of carryover effect in our research is that after having seen a smiling person, the good mood produced by the smile might last for a while, affecting people’s judgments of later stimuli … even those in the non-smiling condition. There is no special name for this. Just be aware that carryover effects come in many forms.
The problem with carryover effects is that they can become confounding variables. Imagine our famous experiment on the effects of noise on concentration. Imagine further that we test all subjects in the noisy condition followed by the quiet one and that they perform better under the quiet condition. But perhaps this is just a practice effect. Maybe they performed better under the second condition not because it quiet, but because they had practice at the concentration task. Or imagine that there were no apparent effect of noise. Maybe this is because they had become fatigued by the time they were in the quiet condition.
One solution to the problem of carryover effects is to counterbalance the order of your conditions—that is, to test different subjects under the different conditions in different orders. For example, you might test some subjects under the quiet condition and then under the noisy condition, and the rest under the noisy condition and then under the quiet condition. How would you decide which subjects to test in which order? You guessed it! Random assignment (either with our without matching).
Why does counterbalancing help? In many cases, the carryover effect in one direction will simply cancel out the carryover effect in the other direction. Imagine that there is a practice effect in our noise experiment, so that subjects tend to be better at the concentration task under the second condition. If we counterbalance, then for some subjects the practice effect will boost their performance a bit in the noisy condition, and for others it will boost their performance a bit in the quiet condition. As a result, these two effects will cancel each other out when we aggregate the data across all the subjects.
Sometimes, though, there is a carryover effect in one direction but not the other. For example, my group size effect research has shown that if subjects rate the average member of a group and then rate the individuals in that group, they rate the average member of the group at higher risk than most individuals. But if they rate the individuals first and then the average member of the group, this effect goes away. It appears that when they make the ratings in this order, they rate the average member of the group by mentally averaging the ratings they already made for the individuals. This could happen with smiling too. It could be that a smiling stimulus person judged after a non-smiling one is judged more favorably, but a smiling stimulus person judged before a non-smiling one is not. Again, this could be because the good mood produced by the smiling stimulus person carries over to the non-smiling one. Although these carryover effects does not cancel out across the different orders, it is possible to analyze the data separately for the two orders and to see the carryover effect in action.
Counterbalancing More Than Two Conditions
With just two conditions, counterbalancing is easy. You test some subjects in Condition A followed by Condition B, and the rest in Condition B followed by Condition A. If there are three conditions, however, there are six different orders: ABC, ACB, BCA, BAC, CAB, CBA. If there are four conditions, then there are 24 different orders! (How many are there with five conditions?) Standard practice is to use all the different orders whenever possible. As you can imagine, there are situations in which it is not feasible to use all the different orders. There are special techniques to handle these situations, but they are not that important for our purposes.
When Within-Subjects Designs Do Not Work
Sometimes, within-subjects designs pose problems that cannot be solved by counterbalancing or any other technique. In a study of psychotherapy effectiveness, if you assign some subjects to the treatment group first and you cure them, you cannot then put them in a control group. The treatment has permanently changed them in a way that makes them unsuitable for further study. This kind of study would require a randomized between-subjects design
Between-Subjects versus Within-Subjects Designs
In general, researchers in psychology seem to have a preference for within-subjects designs because of their greater efficiency. So it is probably best for you to have a bias toward using within-subjects designs when possible. There may be situations, though, in which it seems best to avoid carryover effects altogether by using a between-subjects design. For example, in my own work, I find that the desire to keep the subjects from “catching on” to the hypothesis is one of the most common reasons for using a between-subjects manipulation.