Carryover Effects and Counterbalancing

Carryover Effects

The problem with within-subjects designs is that they can result in carryover effects.  This is when having been tested under one condition affects how participants behave in another condition.  There are many different kinds of carryover effects.  Here are a few of the most important.

1. Practice and Fatigue Effects – These occur when subjects get better at the task over time because of practice or get worse at the task over time because they get tired.

2. Assimilation and Contrast Effects – These occur when a stimulus is perceived as particularly similar to a preceding stimulus (assimilation) or as particularly different from a preceding stimulus (contrast).  Imagine, for example, a study of how teachers grade essays. If a poor essay is presented after a good essay, the poor essay might be judged as better than it would have been on its own ... especially if it was thought to have been written by the same student. (Teacher: "This student already wrote one good essay, so this one must be pretty good too.")   Alternatively, the poor essay might be judged as worse than it would have been on its own ... especially if it was thought to have been written by a different student. ("Compared to the last one, this one is especially terrible.") An example of a contrast effect is that a noisy condition experienced after a quiet condition might be perceived as even noisier than it normally would be.  Or a stimulus person who is not smiling presented after a stimulus person who is smiling might be perceived as especially unfriendly.

3. “Catching On” Effects – This is a non-standard term that describes situations in which being in more than one condition makes it clear to subjects what the independent variable is, so that they “catch on” to the hypothesis being tested.  In many cases, subjects will then behave the way they think you want them too.  Some disagreeable subjects might behave just the opposite!

The Problem with Carryover Effects

The problem with carryover effects is that they can become confounding variables.  Imagine that in an experiment on the effect of noise on concentration we test all subjects in the noisy condition followed by the quiet condition. Imagine further than onn average the subjects perform better in the quiet condition.  Is this an effect of the noise level? Maybe. But it might also be a practice effect. Maybe subjects performed better under the second condition not because it was quiet, but because they had a chance to practice in the first condition.

Counterbalancing

One solution to the problem of carryover effects is to counterbalance the order of your conditions—that is, to test different subjects under the different conditions in different orders.  For example, you might test some subjects under the quiet condition followed by the noisy condition, and the rest under the noisy condition followed by the quiet condition.  How would you decide which subjects to test in which order?  You guessed it!  Random assignment (either with our without matching).

Why does counterbalancing help?  In many cases, the carryover effect in one direction will simply cancel out the carryover effect in the other direction.  Imagine that there is a practice effect in our noise experiment, so that subjects tend to be better at the concentration task under the second condition.  If we counterbalance, then for some subjects the practice effect will boost their performance a bit in the noisy condition, and for others it will boost their performance a bit in the quiet condition.  As a result, these two effects will cancel each other out when we aggregate the data across all the subjects.

Sometimes, though, there is a carryover effect in one direction but not the other.  For example, Dr. Price's risk judgment research has shown that if subjects rate risk of the average member of a group and then rate the risk of the individuals in that group, they rate the average member of the group at higher risk than most individuals.  But if they rate the individuals first and then the average member of the group, this effect goes away.  It appears that when they make the ratings in this order, they rate the average member of the group by mentally averaging the ratings they already made for the individuals.  Although this carryover effects does not cancel out across the different orders, it is possible to analyze the data separately for the two orders and to see the carryover effect in action.

Counterbalancing More Than Two Conditions

With just two conditions, counterbalancing is easy.  You test some subjects in Condition A followed by Condition B, and the rest in Condition B followed by Condition A.  If there are three conditions, however, there are six different orders: ABC, ACB, BCA, BAC, CAB, CBA.  If there are four conditions, then there are 24 different orders!  (How many are there with five conditions?)  Standard practice is to use all the different orders whenever possible.  As you can imagine, there are situations in which it is not feasible to use all the different orders.  There are special techniques to handle these situations. Although we do not cover these techniques here, we feel confident that if you understand the basic principles of counterbalancing, you can understand these techniques on your own should you ever need them.

When Within-Subjects Designs Do Not Work

Sometimes, within-subjects designs pose problems that cannot be solved by counterbalancing or any other technique.  In a study of psychotherapy effectiveness, if you assign some subjects to the treatment group first and they improve, you cannot then put them in a control group.  The treatment has permanently changed them in a way that makes them unsuitable for further study.  This kind of study would require a randomized design.

Between-Subjects versus Within-Subjects Designs

In general, researchers in psychology seem to have a preference for within-subjects designs because of their greater efficiency.  So it is probably best for you to have a bias toward using within-subjects designs when possible.  There may be situations, though, in which it seems best to avoid carryover effects altogether by using a between-subjects design.  For example, in our research we find that the desire to keep the subjects from “catching on” to the hypothesis often leads us to use between-subjects designs.