Loading The Dice: How Not To Do Research

Here are six ways to load the dice.

1. Choose a biased sample. If you want to "prove" that Americans prefer Chrysler over Toyota's, interview unemployed union workers who trace their job loss to Japanese imports. The answer is predictable. You'll get what you're looking for.


2. Ask biased questions. Even if you choose an unbiased sample, you can phrase questions in such a way that you direct people to the answer you're looking for.


3. List biased choices. Another way to load the dice is to use closed-ended questions that push people into the answer that you want.


4. Discard undesirable results. Researchers can keep silent about results they find embarrassing, or they can continue to survey samples until they find one that matches what they were looking for.


5. Misunderstand the subjects' world. This route can lead to errors every bit as great as those just cited. Even researchers who use good samples, word their questions properly, and offer adequate choices can end up with skewed results. They may, for example, fail to anticipate that people may be embarrassed to express an opinion that isn't politically correct.


6. Analyze the data incorrectly. Even when researchers strive for objectivity, the sample is good, the wording is neutral, and the respondents answer the questions honestly, the results can still be skewed. The researchers may make mistake in their calculations, such as entering incorrect date into the computers. This too is inexcusable in science.

Dig This?